extended import and callback API
extended import and callback API
The import documentation in the reference manual has been rewritten with a second chapter for the extended import FFI API. Any help in improving language and punctuation is appreciated. Simply copy paste corrected paragraphs in this thread and I will integrate them.
http://www.newlisp.org/downloads/develo ... mport.html
Special thanks again to sunmountain for implementing the extended import syntax.
The new extended syntax will work automatically for both the 32-bit and 64-bit versions of newLISP automatically and allows feeding and returning most of C's basic data types. The new API is more practical when porting code from other scripting languages. The libffi is part of standard Mac OS X and Windows 7 and XP and UBUNTU linux installs. On other UNIX libffi is easy to make and add.
The old API, which is used in existing standard modules, is preferred in embedded applications when trying to install a minimum amount of libraries and when speed performance is critical.
sunmountain is currently working on an extended FFI callback API. As soon as this is integrated a new development version 10.3.8 will be released.
ps: CodePatterns.html will be updated later
http://www.newlisp.org/downloads/develo ... mport.html
Special thanks again to sunmountain for implementing the extended import syntax.
The new extended syntax will work automatically for both the 32-bit and 64-bit versions of newLISP automatically and allows feeding and returning most of C's basic data types. The new API is more practical when porting code from other scripting languages. The libffi is part of standard Mac OS X and Windows 7 and XP and UBUNTU linux installs. On other UNIX libffi is easy to make and add.
The old API, which is used in existing standard modules, is preferred in embedded applications when trying to install a minimum amount of libraries and when speed performance is critical.
sunmountain is currently working on an extended FFI callback API. As soon as this is integrated a new development version 10.3.8 will be released.
ps: CodePatterns.html will be updated later
Re: extended import and callback API
This is a great addition to newLISP!!!
Some quick thoughts:
-- xytroxon
Some quick thoughts:
Code: Select all
; An idea for dlls with multiple functions. To allow for a list
; in the second argument to define a dll's functions as a group.
; So to improve readability and reduce typing errors when
; adding new functions from header source code and
; for easier testing of multiple dll versions...
; (import MY_DLL
(import MY_DLL_NEW
'(
;; void func1(int var1);
("func1" "void" "int")
;; int func2(char var2);
("func2" "int" "char")
; etc...
)
)
; That caused a second idea, it would look better to
; retain the original C header code format i.e.
; return variable / function name / variable(s).
; The purpose: Tto reduce errors transposing C function
; headers from copy and paste editing
(import MY_DLL
'(
;; void func1(int var1);
("void" "func1" "int")
;; int func2(char var2, int *var3);
("int" "func2" "char" "int*")
; etc...
)
)
"Many computers can print only capital letters, so we shall not use lowercase letters."
-- Let's Talk Lisp (c) 1976
-- Let's Talk Lisp (c) 1976
Re: extended import and callback API
Here is a better way, that doesn't need to mess with import...
-- xytroxon
Code: Select all
(define (import-ffi libname funclist)
(dolist (func funclist)
(eval (push import (push libname func)))
)
)
(import-ffi "msvcrt.dll"
'(
("atof" "double" "char*")
("atoi" "int" "char*")
; etc...
)
)
"Many computers can print only capital letters, so we shall not use lowercase letters."
-- Let's Talk Lisp (c) 1976
-- Let's Talk Lisp (c) 1976
Re: extended import and callback API
I can put that into CodePatterns.html.
-
- Posts: 39
- Joined: Tue Mar 15, 2011 5:11 am
Re: extended import and callback API
Hi,
Yes I'm done with implementing the closure callback extention.
Now you can write code like this:
which gives (your rng may create alternate numbers, though):
And you can (for instance) interact in more "natural" way with ui libraries.
For example I use IUP very often.
To use it, I place a iup.dll in the same directory as newlisp.exe and then do this:
If run and clicking on the first or second button gives something like this:
As a sideeffect of this extension the number of callbacks is virtually unlimited.
What is missing is support for varargs (which is really hard to implement in ffi context,
and support for structs/unions, which is not so hard to implement.
By now the extension adds ca. 40 kB (Windows XP SP2, gcc 4.6.1 (mingw)).
I'm looking forward to see this in 10.3.8 :-)
And again: a big thank you to lutz for this opportunity.
Yes I'm done with implementing the closure callback extention.
Now you can write code like this:
Code: Select all
(import "msvcrt" "qsort" "void" "void*" "int" "int" "void*")
(import "msvcrt" "atexit" "void" "void*")
(define (bye) (println "See you ..."))
(atexit (callback 'bye "void" "void"))
(set 'l '())
(dotimes (i 30)
(set 'l (append l (list (int (mul 30 (random)))))))
(set 'base (pack (dup "ld " 30) l))
(println "Unsorted:")
(println (unpack (dup "ld " 30) base))
(define (cmp a b)
(- (get-int a) (get-int b)))
(qsort (address base) 30 4 (callback 'cmp "int" "void*" "void*"))
(println "Sorted by qsort:")
(println (unpack (dup "ld " 30) base))
(exit)
Code: Select all
Unsorted:
(0 16 5 24 17 14 10 26 24 22 5 25 21 15 9 0 2 10 4 4 29 13 3 0 0 11 15 17 18 18)
Sorted by qsort:
(0 0 0 0 2 3 4 4 5 5 9 10 10 11 13 14 15 15 16 17 17 18 18 21 22 24 24 25 26 29)
See you ...
For example I use IUP very often.
To use it, I place a iup.dll in the same directory as newlisp.exe and then do this:
Code: Select all
(import "iup.dll" "IupOpen" "void" "void")
(import "iup.dll" "IupClose" "void" "void")
(import "iup.dll" "IupButton" "int" "char*" "int")
(import "iup.dll" "IupSetAttribute" "int" "int" "char*" "char*")
(import "iup.dll" "IupSetAttributeHandle" "int" "int" "char*" "void*")
(import "iup.dll" "IupSetCallback" "int" "int" "char*" "void*")
(import "iup.dll" "IupShowXY" "int" "int" "int" "int")
(import "iup.dll" "IupDialog" "int" "int")
(import "iup.dll" "IupMainLoop" "int" "void")
(import "iup.dll" "IupLabel" "int" "char*")
(import "iup.dll" "IupVbox" "int" "int" "int" "int" "int")
(import "msvcrt.dll" "atexit" "void" "void*")
(IupOpen)
(set 'quit_bt (IupButton "Quit" 0))
(set 'quit_bt2 (IupButton "More Quit" 0))
(define (quit_cb handle button pressed x y status)
(begin
(println "handle id " handle)
(if (= handle quit_bt) (println "quit_bt pressed "))
(if (= handle quit_bt2) (println "quit_bt2 pressed "))
(println "pressed? " (if (= 1 pressed) "Yes" "No"))
(println "x " x)
(println "y " y)
(println "status " (get-string status))
-2) ; IUP_DEFAULT = -2
)
(define (end) (println "Quit !"))
(IupSetCallback quit_bt "BUTTON_CB" (callback 'quit_cb "int" "int" "int" "int" "int" "int" "char*"))
(IupSetCallback quit_bt2 "BUTTON_CB" (callback 'quit_cb "int" "int" "int" "int" "int" "int" "char*"))
(set 'label (IupLabel "Very long label"))
(IupSetAttribute label "EXPAND" "YES")
(IupSetAttribute label "ALIGNMENT" "ACENTER")
(set 'vbox (IupVbox label quit_bt quit_bt2 0))
(IupSetAttribute vbox "MARGIN" "10x10")
(IupSetAttribute vbox "GAP" "5")
(IupSetAttribute vbox "ALIGNMENT" "ACENTER")
(set 'dialog (IupDialog vbox) )
(IupSetAttribute dialog "EXPAND" "YES")
(IupSetAttribute dialog "SIZE" "QUARTER")
(IupSetAttributeHandle dialog "DEFAULTESC" quit_bt)
(IupShowXY dialog 100 100)
(IupSetAttribute dialog "TITLE" "IUP from newLisp 10.3.8_DEVEL")
(atexit (callback 'end "void" "void"))
(IupMainLoop)
(IupClose)
(exit)
Code: Select all
$ newlisp.exe iup_button.lsp
handle id 23449624
quit_bt pressed
pressed? Yes
x 22
y 9
status 1
handle id 23449624
quit_bt pressed
pressed? No
x 22
y 9
status 1
handle id 23449624
quit_bt pressed
pressed? Yes
x 22
y 9
status 3
handle id 23449624
quit_bt pressed
pressed? No
x 22
y 9
status 3
handle id 23451248
quit_bt2 pressed
pressed? Yes
x 24
y 17
status 1
handle id 23451248
quit_bt2 pressed
pressed? No
x 24
y 17
status 1
handle id 23451248
quit_bt2 pressed
pressed? Yes
x 24
y 17
status 3
handle id 23451248
quit_bt2 pressed
pressed? No
x 24
y 17
status 3
Quit !
What is missing is support for varargs (which is really hard to implement in ffi context,
and support for structs/unions, which is not so hard to implement.
By now the extension adds ca. 40 kB (Windows XP SP2, gcc 4.6.1 (mingw)).
I'm looking forward to see this in 10.3.8 :-)
And again: a big thank you to lutz for this opportunity.
-
- Posts: 608
- Joined: Mon Feb 05, 2007 1:04 am
- Location: Abbotsford, BC
- Contact:
Re: extended import and callback API
Lutz, have you considered joining the Google Summer of Code?
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.
Re: extended import and callback API
To be a Google Summer Code mentor is a lot of work and also requires a bigger project or a project in earlier stages with more people involved at the beginning.
Re: extended import and callback API
After some struggeling on Mac OS X, the extended ffi with unlimited callbacks is now working on Mac OS X, UBUNTU Linux and Win32.
There is still some minor cleanup for memory management when functions get imported repeatedly in the same newLISP session and some refactoring is in process too. All that should be done soon for an official development release 10.3.8 during this week.
On Mac OSX 10.3.8 with libffi only adds about 10k compared to 10.3.3. On UBUNTU Linux the difference is only about 5k.
There is still some minor cleanup for memory management when functions get imported repeatedly in the same newLISP session and some refactoring is in process too. All that should be done soon for an official development release 10.3.8 during this week.
On Mac OSX 10.3.8 with libffi only adds about 10k compared to 10.3.3. On UBUNTU Linux the difference is only about 5k.
Re: extended import and callback API
After some struggling on Mac OS X, the extended ffi with unlimited callbacks is now working on Mac OS X, UBUNTU Linux and Win32.
There is still some minor cleanup for memory management when functions get imported repeatedly in the same newLISP session and some refactoring is in process too. All that should be done soon for an official development release 10.3.8 during this week.
On Mac OSX 10.3.8 with libffi only adds about 10k compared to 10.3.3. On UBUNTU Linux the difference is only about 5k.
There is still some minor cleanup for memory management when functions get imported repeatedly in the same newLISP session and some refactoring is in process too. All that should be done soon for an official development release 10.3.8 during this week.
On Mac OSX 10.3.8 with libffi only adds about 10k compared to 10.3.3. On UBUNTU Linux the difference is only about 5k.
-
- Posts: 39
- Joined: Tue Mar 15, 2011 5:11 am
Re: extended import and callback API
Hi lutz,
I made a small fix, sent you the code.
Under Windows (mingw, gcc 4.6.1, Win XP Pro SP 2 32 But) it adds
280590-274958 = 5632 Bytes, so the addition is neglectable I think.
But to be fair: it is compute intensive, as it creates call information on the fly using assembler
ninja technics.
Now I can put python to rest :-)
At least, as soon I implemented proper struct handling.
I made a small fix, sent you the code.
Under Windows (mingw, gcc 4.6.1, Win XP Pro SP 2 32 But) it adds
280590-274958 = 5632 Bytes, so the addition is neglectable I think.
But to be fair: it is compute intensive, as it creates call information on the fly using assembler
ninja technics.
Now I can put python to rest :-)
At least, as soon I implemented proper struct handling.
Re: extended import and callback API
The difference of 10.3.8 with extended FFI versus only simple FFI on Mac OSX Lion is:
285248 - 279976 = 5272 bytes
On Linux Ubuntu 11.04:
267072 - 262848 = 4224 bytes
That is less than 2% on the three platforms Windows, OS X and Linux.
For that < 2% we get the ability to import virtually any library.
Hopefully this will inspire more newLISP users to write many extensions ;-)
285248 - 279976 = 5272 bytes
On Linux Ubuntu 11.04:
267072 - 262848 = 4224 bytes
That is less than 2% on the three platforms Windows, OS X and Linux.
For that < 2% we get the ability to import virtually any library.
Hopefully this will inspire more newLISP users to write many extensions ;-)