Calling OBJC runtime from newlisp

Machine-specific discussion
Unix, Linux, OS X, OS/2, Windows, ..?
Locked
fdb
Posts: 66
Joined: Sat Nov 09, 2013 8:49 pm

Calling OBJC runtime from newlisp

Post by fdb »

I'm trying to call the objc runtime from new lisp (from mac osx10.9) but i've got a problem:

Code: Select all

> (set 'get-class (import "libobjc.dylib" "objc_getClass" "void*" "char*"))
objc_getClass@7FFF9391BF70
> (get-class "NSString")
0
What am i doing wrong here? I was expecting a pointer to the NSString class. The definition of the c function is defined in Apple's Objective-C Runtime Reference:https://developer.apple.com/library/mac ... c_getClass
objc_getClass
Returns the class definition of a specified class.

id objc_getClass(const char *name)
Parameters
name
The name of the class to look up.
Return Value
The Class object for the named class, or nil if the class is not registered with the Objective-C runtime.
I got the idea from this blog post http://sjhannah.com/blog/?p=219 which calls the objective-C runtime from Java by using the c-funtions:
objc_msgSend(), to pass messages to objects,
objc_getClass(), to get a pointer to a class and
selector_getUid(), to get the pointer to a selector.
Last edited by fdb on Wed Sep 03, 2014 3:26 pm, edited 1 time in total.

fdb
Posts: 66
Joined: Sat Nov 09, 2013 8:49 pm

Re: Callig OBJC runtime from newlisp

Post by fdb »

Was probably wrong to use a literal string but this also doesn't work:

Code: Select all

> (set 'nsstring "NSString") 
"NSString"
> (get-class nsstring)
0
> 

fdb
Posts: 66
Joined: Sat Nov 09, 2013 8:49 pm

Re: Callig OBJC runtime from newlisp

Post by fdb »

Ok, to answer my own question ;-)

Code: Select all

> (import "/System/Library/Frameworks/Foundation.framework/Foundation")
true
> (get-class "NSString")
140735249161920
Is anyone interested in an newlisp-OBJC bridge? An obvious advantage would be the ability to make native MAC GUI applications, and maybe even IOS applications but i don't know how feasible/desirable that is. I'm going to see how far i can take this, but i don't have a lot of experience in C/ObjC/Cocoa and any help especially regarding memory mgt and proxy's is very welcome!

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Re: Callig OBJC runtime from newlisp

Post by TedWalther »

Right on! Please keep us updated!
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.

fdb
Posts: 66
Joined: Sat Nov 09, 2013 8:49 pm

Re: Callig OBJC runtime from newlisp

Post by fdb »

Ok, got the basic machinery working to send messages to classes and instances, see below. Next step is to use callbacks for setting delegate's so you could populate a table view from newLISP.

Code: Select all

;;Objective-c bridge

(import "/System/Library/Frameworks/Foundation.framework/Foundation")

(set 'get-class (import "libobjc.dylib" "objc_getClass"))

(set 'get-sel 	(import "libobjc.dylib" "sel_getUid"))

(set 'msg (import "libobjc.dylib" "objc_msgSend"))

;; principal function sending a message to an instance or class with a variable 
;; number of arguments
(define (send-to ptr sel)
	(eval (flat (list 'msg ptr (get-sel sel) (args)))))

;; convenience functions can also be made for all classes which will let
;; code look exactly like OBJC syntax (without the ugly square brackets ofcourse)

(define (NSString sel)	
	(send-to (get-class "NSString") sel (args)))

;; we can then send messsages to classes which almost looks like standard OBJC syntax
;; at least for one variable statements

(set 'my-ns-string(NSString "stringWithUTF8String:" "i'm a pointer to an NSString instance"))

;; converting back from ns-string to normal string, as this is a pointer you need to 
;; convert back with get-string. We are sending it to the instance we created with
;; send-to but first make a shortcut to make it almost look like OBJC

(set (sym "_") send-to)

(get-string (_ my-ns-string "UTF8String"))

true
objc_getClass@7FFF9391BF70
sel_getUid@7FFF93920693
objc_msgSend@7FFF9391C080
(lambda (ptr sel) (eval (flat (list 'msg ptr (get-sel sel) (args)))))
(lambda (sel) (send-to (get-class "NSString") sel (args)))
4296018608
(lambda (ptr sel) (eval (flat (list 'msg ptr (get-sel sel) (args)))))
"i'm a pointer to an NSString instance"

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Re: Calling OBJC runtime from newlisp

Post by TedWalther »

This is very exciting work.

Also, last night I pulled in some libc functions on my linux box; utimes(2) and link(2). It was so fast and easy I had a hard time believing it Just Worked. Thanks to Lutz for the libffi stuff, and the guy who came up with the (struct ..) function.
(import "/lib/x86_64-linux-gnu/libc.so.6" "utime" "int" "char*" "void*")
(import "/lib/x86_64-linux-gnu/libc.so.6" "link" "int" "char*" "char*")
(struct 'utimbuf "long" "long")

(link "file1" "file2")
(when (!= (file-info "file1" 6) (file-info "file3" 6))
(utime "file3" (pack utimbuf (select (file-info "file1") 5 6))))
Only took 5 minutes. newLisp is letting me get over that nervous feeling of "what is going to go wrong next" every time I try something new.
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.

fdb
Posts: 66
Joined: Sat Nov 09, 2013 8:49 pm

Re: Calling OBJC runtime from newlisp

Post by fdb »

I'm having problems with a function which returns a struct. To make a window appear on the screen i need an NSRect type, which is defined as a struct: {{CGFloat,CGFloat},{CGFloat,CGFloat}}. I've imported this function:

Code: Select all

(import "/System/Library/Frameworks/Foundation.framework/Foundation" "NSRectFromString")

NSRectFromString@7FFF8C0561F9
'; but when called

(NSRectFromString "1 2 3 4")
There is nothing, Newlisp halts and i need to restart the shell.

Is there another way to construct this struct and pass it to another function? I've tried

Code: Select all

(struct 'point "double" "double")
(struct 'size "double" "double")
(struct 'rect "point" "size")
(set 'my-rect (pack rect (pack point 100.0 100.0) (pack size 100.0 100.0)))
and pass my-rect but doesn't work

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Re: Calling OBJC runtime from newlisp

Post by TedWalther »

NSRectFromString is expecting an NSString class as an argument, but you are passing it a pointer to a regular C string.

Wish I had access to the header files and stuff where this is implemented. Trying to find the class definition of NSString was tough; how would you use (struct) in newlisp to create an NSString. Or let's say you call the NSString() initializer that creates it for you, how do you modify its members inside newLISP.
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.

fdb
Posts: 66
Joined: Sat Nov 09, 2013 8:49 pm

Re: Calling OBJC runtime from newlisp

Post by fdb »

Hi Ted,

you're right NSRecFromString is expecting an NSString, copied the wring line but

Code: Select all

(NSRectFromString (NSString "stringWithUTF8String:" "100,100,100,100"))
doesn't work either. I think it is because it is returning a struct by value and in the import function you cannot define that it returns a struct by value, only pointers, primitive types and c-strings. Not sure if this is a limitation in the libffi library or just not implemented in newLisp.

I found a very nice toolset at ww.f-script.org which does what i try to achieve in newlIsp. Looks like they package structs in NSValue instances and have their own functions written in objective-c but i was hoping that i could interact with the runtime without an intermediate layer.

fdb
Posts: 66
Joined: Sat Nov 09, 2013 8:49 pm

Re: Calling OBJC runtime from newlisp

Post by fdb »

Solved the returning of a struct

Code: Select all

(struct 'Point "double" "double")
(struct 'Rect "point" "point")
(import "/System/Library/Frameworks/Foundation.framework/Foundation" "NSRectFromString" "Rect" "void*")
So you can define the struct as a return type, was not mentioned in the documentation for import , so please add this.

Code: Select all

(setq myrect (NSRectFromString (@@ "100 200 50 20")))
((100 200) (50 20))

fdb
Posts: 66
Joined: Sat Nov 09, 2013 8:49 pm

Re: Calling OBJC runtime from newlisp

Post by fdb »

I've changed my message sending function to • (alt 8) and can alternate in methods and value and it looks nice have imported some constants and such and works fine except for structs..(i think) i can't get the following code to work

Code: Select all

(setq window (•(• NSWindow "alloc") "initWithContentRect:" (@rect "0  0 200 200") "styleMask:" NSTitledWindowMask "backing:" NSBackingStoreBuffered "defer:" NO))												
(• window "orderFront:" nil) 				
2014-09-14 19:58:48.249 newlisp[1499:507] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Error (1000) creating CGSWindow'
*** First throw call stack:
(
0 CoreFoundation 0x00007fff87c5025c __exceptionPreprocess + 172
1 libobjc.A.dylib 0x00007fff82d90e75 objc_exception_throw + 43
2 CoreFoundation 0x00007fff87c5010c +[NSException raise:format:] + 204
3 AppKit 0x00007fff89d0cf81 _NXCreateWindowWithStyleMask + 366
4 AppKit 0x00007fff89d0cd7d _NSCreateWindow + 64
5 AppKit 0x00007fff89bc572c -[NSWindow _commonAwake] + 2963
6 AppKit 0x00007fff89bd3b63 -[NSWindow _reallyDoOrderWindow:relativeTo:findKey:forCounter:force:isModal:] + 1121
7 AppKit 0x00007fff89bd3460 -[NSWindow _doOrderWindow:relativeTo:findKey:forCounter:force:isModal:] + 786
8 AppKit 0x00007fff89bd30e0 -[NSWindow orderWindow:relativeTo:] + 162
9 newlisp 0x00000001000262c0 newlisp + 156352
10 newlisp 0x000000010000539b newlisp + 21403
11 newlisp 0x000000010000667a newlisp + 26234
12 newlisp 0x000000010000530f newlisp + 21263
13 newlisp 0x0000000100005e33 newlisp + 24115
14 newlisp 0x000000010000530f newlisp + 21263
15 newlisp 0x0000000100009006 newlisp + 36870
16 newlisp 0x000000010000537e newlisp + 21374
17 newlisp 0x0000000100009e6d newlisp + 40557
18 newlisp 0x000000010000b3c2 newlisp + 46018
19 newlisp 0x000000010000bc57 newlisp + 48215
20 newlisp 0x0000000100000d3c newlisp + 3388
)
libc++abi.dylib: terminating with uncaught exception of type NSException
The runtime also complains about the style mask, but the message can differ every time i invoke the init method.
Lutz could you help here? After some googling i found that it may be the FFI: big structs 'are not passed on the stack' or something, i'm really out of my depth here and any help is much appreciated.

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Re: Calling OBJC runtime from newlisp

Post by Lutz »

The fact, that the error looks different each time, makes me think that already reclaimed memory is overwritten.

I am not at all familiar with Objective C and how it manages memory but could imagine, that some of the memory allocated for structures by Objc—C (coming back as structure return values) is collected soon afterwards by Objective C. What might help is: not to nest newLISP expressions too much but do assignments first. This way memory crated in Obc-C functions gets copied/created and managed inside newLISP. E.g.:

Code: Select all

(setq theRect (@rect "0  0 200 200"))
(setq dotAlloc (• NSWindow "alloc"))
(setq window (• dotAlloc  "initWithContentRect:" theRect "styleMask:" NSTitledWindowMask "backing:" NSBackingStoreBuffered "defer:" NO))
.. but this is just a wild guess ;)

Ps: I assume that @rect and • are functions imported from Objective C
Ps: on second thought if @rect and • return just some integer handle numbers/addresses it won't make a difference and still crash

fdb
Posts: 66
Joined: Sat Nov 09, 2013 8:49 pm

Re: Calling OBJC runtime from newlisp

Post by fdb »

Well i'm almost there (i hope)
The crashing is because you need to instantiate an NSApplication first which generates an event loop and takes care of events in the Window. So this tiny ObjC program works fine, the application starts and displays and empty window.

Code: Select all

#import <AppKit/AppKit.h>
#import <Foundation/Foundation.h>
int main ()
{
    [NSApplication sharedApplication];
    [NSApp setActivationPolicy:NSApplicationActivationPolicyRegular];
    NSRect myrect = NSRectFromString(@"20 20 200 200");
    id window = [[NSWindow alloc]   initWithContentRect:myrect
                                              styleMask:3
                                                backing:NSBackingStoreBuffered
                                                  defer:NO];
    
    [window orderFront:nil];
    [NSApp activateIgnoringOtherApps:YES];
    [NSApp run];
    return 0;
}
However my newLisp program which (should do) the same starts the application fine, but it doesn't draw a window (or it is so tiny i can't see it...

Code: Select all

(setq nsapp (• NSApplication "sharedApplication"))
(• nsapp "setActivationPolicy:" NSApplicationActivationPolicyRegular)
(set 'arect (@rect "20 20 200 200"))
(set 'amask NSTitledWindowMask)
(set 'abacking NSBackingStoreBuffered)
(set 'adefer NO)
(setq awind (• NSWindow "alloc"))
(setq window (• awind "initWithContentRect:" arect
       					"styleMask:" amask
								"backing:" NSBackingStoreBuffered
								"defer:" adefer))
(• window "makeKeyAndOrderFront:" nil)
(• nsapp "activateIgnoringOtherApps:" YES)
(• nsapp "run")
I still suspect that the rect struct isn't passed correctly, for the other values i also tested with 'packing' them but that didn't help either, Does anyone know how to test that the rect struct is passed correctly?

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Re: Calling OBJC runtime from newlisp

Post by Lutz »

Code: Select all

    [window orderFront:nil];
    [NSApp activateIgnoringOtherApps:YES];
and

Code: Select all

(• window "makeKeyAndOrderFront:" nil)
(• nsapp "activateIgnoringOtherApps:" YES)
for sure nil and YES are not the same in newLISP and Objective C. In case of YES the contents of the newLISP variable YES will be passed. Somewhere you would have to set YES to the same value it has in Objective C. Probably you can find out from the header files. The same is true for nil, you probably have to define a NIL in newLISP for usage with Objective C:

Code: Select all

(set 'NIL <whatever Obj-C wants here>) ; for usage with Obj-C
I guess somewhere in the Obj-C docs or header files you can find the value.

fdb
Posts: 66
Joined: Sat Nov 09, 2013 8:49 pm

Re: Calling OBJC runtime from newlisp

Post by fdb »

Hi Lutz , yes i've defined them in newLisp, they're enum's. So to clarify this works:

Code: Select all

#import <AppKit/AppKit.h>
#import <Foundation/Foundation.h>
int main ()
{
    [NSApplication sharedApplication];
    [NSApp setActivationPolicy:NSApplicationActivationPolicyRegular];
    NSRect myrect = NSRectFromString(@"20 20 200 200");
    id window = [[NSWindow alloc]   initWithContentRect:myrect
                                              styleMask:1
                                                backing:2
                                                  defer:0];
    
    [window orderFront: window];
    [NSApp activateIgnoringOtherApps:1];
    [NSApp run];
    return 0;
}
and the same in newLisp doesn't

Code: Select all

(setq nsapp (• NSApplication "sharedApplication"))
(• nsapp "setActivationPolicy:" NSApplicationActivationPolicyRegular)
(set 'rect (@rect "20 20 200 200"))
(set 'mask NSTitledWindowMask)
(set 'backing NSBackingStoreBuffered)
(set 'defer 0)
(setq awind (• NSWindow "alloc"))
(setq window (• awind "initWithContentRect:" rect "styleMask:" 1 "backing:" 2 "defer:" 0))
(• window "orderFront:" window)
(• nsapp "activateIgnoringOtherApps:" 1)
(• nsapp "run")

fdb
Posts: 66
Joined: Sat Nov 09, 2013 8:49 pm

Re: Calling OBJC runtime from newlisp

Post by fdb »

Well i got it working! However structs are not passed correctly by libffi or the objc runtime is to blame. In order to get past this I made a small objc dynamic library:

Code: Select all


#import "bridge_newlisp_objc.h"

id create_window (id window , char * rect , NSUInteger mask ){
    NSRect myrect = NSRectFromString([NSString stringWithUTF8String:rect]);
    return [window initWithContentRect:myrect
                      styleMask:mask
                        backing:2
                          defer:NO];   
}
And know i can call the rest from new lisp as follows:

Code: Select all

(load "~/Documents/newlisprogjes/Modules/objc-bridge.lsp")

(setq nsapp (• NSApplication "sharedApplication"))
(• nsapp "setActivationPolicy:" NSApplicationActivationPolicyRegular)
(setq window (• NSWindow "alloc"))
(create_window window "200 200 300 300" 15)
(• window "makeKeyAndOrderFront:" window)
(setq title (@s "The Window title"))
(• window "setTitle:" title)
(• nsapp "activateIgnoringOtherApps:" YES)
(• nsapp "run")
and at last i got my window on screen! As you can see i can set the window title afterwards and also numbers are passed correctly only structs not because trying to change the size afterwards by passing a struct doesn't work.

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Re: Calling OBJC runtime from newlisp

Post by TedWalther »

Very nice.
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.

fdb
Posts: 66
Joined: Sat Nov 09, 2013 8:49 pm

Re: Calling OBJC runtime from newlisp

Post by fdb »

Coming back to this (after 4 years , time flies!) I've recently made a minimalistic objective-c/cocoa bridge which doesn't need any 'glue' c-code /libraries. You only need to import 3 functions from the libobjc.dylib and two frameworks (foundation and appkit) which are standard available on a Mac. It is only 22 loc... but gives you the ability to create native GUI's on a Mac with newLisp. In order to use it you need some basic understanding of Objective-C and Cocoa (Google is your friend) but I'm going to start by building a layer on top of this conforming to the same API/functions as guiserver.lsp. So this is what the code looks like:

Code: Select all

;;@module Minimalistic Cocoa / Objective-C bridge

(import "/System/Library/Frameworks/Foundation.framework/Foundation")
(import "/System/Library/Frameworks/AppKit.framework/AppKit")
(set 'get-class (import "libobjc.dylib" "objc_getClass"))
(set 'get-sel (import "libobjc.dylib" "sel_getUid"))
(import "libobjc.dylib" "objc_msgSend")

;; principal function sending a message to an instance or class with a variable 
;; number of arguments use when the arguments are NOT floating points or structs

(define (-> ptr sel)
	(when (string? ptr) (set 'ptr (get-class ptr)))
  (eval (flat (list 'objc_msgSend ptr (get-sel sel) (args))))
)

;; function sending a message to an instance or class using an invocation object,
;; use when the arguments contain floats and or structs. The address of the return 
;; value is returned

(define (=> target selector)	
	(when (string? target)
		(set 'target (get-class target)))
	(set 'sel (get-sel selector))
	(set 'method (-> target "methodSignatureForSelector:" sel))
	(set 'invocation (-> "NSInvocation" "invocationWithMethodSignature:" method))
	(-> invocation "setSelector:" sel)
	(doargs (arg)
		(-> invocation "setArgument:atIndex:" (address arg) (+ $idx 2)))
	(-> invocation "invokeWithTarget:" target)
	(set 'size (-> method "methodReturnLength"))
	(set 'return (dup "\000" size))
	(-> invocation "getReturnValue:" (address return))
	(address return)
)
To see an example see below code, which creates a window with a button on it.

Code: Select all

(struct 'Point "double" "double")
(struct 'Rect "Point" "Point")

(define (makerect x y w h)
	(pack Rect (pack Point x y) (pack Point w h)) )

(define (test)
	(set 'nsapp (-> "NSApplication" "sharedApplication"))
	(-> nsapp "setActivationPolicy:" 1)
	(set 'window (-> "NSWindow" "alloc"))
	(set 'rect (makerect 150 150 200 300))
	(set 'mask 15 'backing 2 'defer 0)
	(=> window "initWithContentRect:styleMask:backing:defer:" rect mask backing defer)
	(set 'button (-> "NSButton" "alloc"))
	(=> button "initWithFrame:" (makerect 100 150 80 40))
	(-> button "setBezelStyle:" 1)
	(-> (-> window "contentView") "addSubview:" button)
	(-> window "makeKeyAndOrderFront:" window)
	(-> nsapp "activateIgnoringOtherApps:" 1)
	(-> nsapp "run")
)


TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Re: Calling OBJC runtime from newlisp

Post by TedWalther »

That is so beautiful it makes me want to cry.
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence. Nine months later, they left with a baby named newLISP. The women of the ivory towers wept and wailed. "Abomination!" they cried.

fdb
Posts: 66
Joined: Sat Nov 09, 2013 8:49 pm

Re: Calling OBJC runtime from newlisp

Post by fdb »

And as probably the final piece I imported the objc function to dynamically add a method (= function) to an existing class with a given implementation (function pointer) which gives you the possibility to execute a newlisp function when for instance clicking on a button. You'll have to define the function first before adding the function to the class. See below code which executes the newlisp function "button-clicked" when you.. click the button! I think I've got everything in place now to copy the functionality of the guiserver for OSX , so no need for Java anymore.

Code: Select all

(define (button-clicked id sel)
	(println "button id:" id " function:" (get-string sel))
)

(define (test)
	(set 'nsapp (-> "NSApplication" "sharedApplication"))
	(-> nsapp "setActivationPolicy:" 1)
	(set 'window (-> "NSWindow" "alloc"))
	(set 'rect (make-rect 150 150 200 300))
	(set 'mask 15 'backing 2 'defer 0)
	(=> window "initWithContentRect:styleMask:backing:defer:" rect mask backing defer)
	(set 'imp (callback 'button-clicked "void" "void*" "void*" ))
	(@add-method (@class "NSButton") (@selector "button-clicked") imp "v@:")
	(set 'button (-> "NSButton" "alloc"))
	(=> button "initWithFrame:" (make-rect 100 150 80 40))
	(-> button "setAction:" (@selector "button-clicked"))
	(-> button "setTarget:" button)
	(-> button "setBezelStyle:" 1)
	(-> (-> window "contentView") "addSubview:" button)
	(-> window "makeKeyAndOrderFront:" window)
	(-> nsapp "activateIgnoringOtherApps:" 1)
	(-> nsapp "run")
)

Locked