Compile script to .exe with attached files

Machine-specific discussion
Unix, Linux, OS X, OS/2, Windows, ..?
Locked
lyl
Posts: 44
Joined: Sun Mar 25, 2018 5:00 am

Compile script to .exe with attached files

Post by lyl »

Source code and the newLISP executable can be linked together to build a self-contained application by using the -x command line flag.
My script file(named "0.lsp" in Windows sys) contains the following code at the begining:

Code: Select all

(load "1.lsp")
(load "2.lsp")
So in this condition how to use "newlisp -x" to build my application which can be used independently in any directory?
I mean, linking all files(0.lsp, 1.lsp, 2.lsp) into a .exe file. Thus, this only one .exe file can work anywhere without any other file to support it.
Last edited by lyl on Mon May 13, 2019 12:20 am, edited 1 time in total.

HPW
Posts: 1390
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

Re: Compile script to .exe with attached files

Post by HPW »

Hello,

I assume that you do want to load your Lisp files from the same Directory (any) as your new exe.
Then you may use (main-args 0) to get the path+name of your running exe.
Extract the path and build full path of your files to load.

Regards
Hans-Peter

lyl
Posts: 44
Joined: Sun Mar 25, 2018 5:00 am

Re: Compile script to .exe with attached files

Post by lyl »

Thank you @HPW. My description in the post made my query not clear. I just modified it. Would you please have look at it?

ralph.ronnquist
Posts: 228
Joined: Mon Jun 02, 2014 1:40 am
Location: Melbourne, Australia

Re: Compile script to .exe with attached files

Post by ralph.ronnquist »

I worked with that notion some while ago. In general it's possible to attach arbitrary data to an exe file, and thus generalize embedding into having multiple files, by overriding some file access functions to use that data where appropriate. In particular you would override load, file? and read-file to use attached data where possible.

This results in two script files: one for packing, and one being a special embedding script that takes care of using the attached data as the original files. For example as follows:

File: buildexe.lsp

Code: Select all

# Requires arguments to be run as:
# newlisp buildexe.lsp lsp1 lsp2 .... binary

(constant 'ARGS (match '(?  "buildexe.lsp" * ?) (main-args)))
(constant 'HEAD "exehead.lsp")

(println "ARGS " ARGS)
(unless ARGS
  (write-line 2 "Run with: newlisp buildexe.lsp lsp1 lsp2 .... binary")
  (exit 1))

(unless (= (! (format "%s -x %s %s" (ARGS 0) HEAD (ARGS 2))))
  (write-line 2 (format "Failed to build %s" (ARGS 2))))

(append-file (ARGS 2) (dup "x" 50))
(dolist (f (ARGS 1))
  (let ((s (string (list 'EmbeddedFiles f (read-file f)))))
    (append-file (ARGS 2) (string (length s) s))))
(append-file (ARGS 2) " nil ")
(append-file (ARGS 2) (string (list 'load (ARGS 1 0))))
(exit 0)
File: exehead.lsp

Code: Select all

(constant 'main_load load)
(constant 'main_read-file read-file)
(constant 'main_file? file?)

(define EmbeddedFiles:EmbeddedFiles nil)

(define (override_load _name)
  (if (EmbeddedFiles _name) (eval-string (EmbeddedFiles _name) MAIN)
    (main_load _name)))

(define (override_read-file _name)
  (if (EmbeddedFiles _name) (EmbeddedFiles _name)
    (main_read-file _name)))

(define (override_file? _name)
  (if (EmbeddedFiles _name) true (main_file? _name)))

(map constant '(load read-file file?)
     (list override_load override_read-file override_file?))

(define SELF:SELF (read-file (or (real-path (main-args 0))
                                 (real-path (string (main-args 0) ".exe")))))

(if (null? SELF)
    (exit (and (write-line 2 (format "No program %s" (main-args 0))) 1))
  (setf override_i (find (dup "x" 50) SELF))
  (let ((override_n 50))
    (while (number?
            (setf override_n
                  (read-expr SELF MAIN nil (inc override_i override_n))))
      (eval (read-expr SELF MAIN nil (inc override_i $count))))
    (eval-string SELF MAIN nil (inc override_i $count)))
  (exit (and (write-line 2 "No program") 1)))
(exit 0)
With those, you use buildexe.lsp for packing your other scripts, with the first being the main script, into a self-contained executable. In actual fact, the executable is simply a newlisp embedding of exehead.lsp with your script files as attached data that gets processed into the EmbeddedFiles hashtable. The filenames are the keys and the contents the values. The overriding functions try to find the given filename in the table first, for overriding, and otherwise fall back on the original functions.

hth

HPW
Posts: 1390
Joined: Thu Sep 26, 2002 9:15 am
Location: Germany
Contact:

Re: Compile script to .exe with attached files

Post by HPW »

Hello,

Just one question: Can't you combine your 3 lsp files into one?
That would be the easiest way.

Regards
Hans-Peter

rickyboy
Posts: 607
Joined: Fri Apr 08, 2005 7:13 pm
Location: Front Royal, Virginia

Re: Compile script to .exe with attached files

Post by rickyboy »

HPW wrote:Hello,

Just one question: Can't you combine your 3 lsp files into one?
That would be the easiest way.

Regards
Good point, HPW.

On another note, I am glad that Ralph posted his scripts which turn loads and file reads (“dynamic inclusion”) into “includes” (“static inclusion”), but only for the files the user indicates during the build invocation. This allows for other files, such as application configuration files, to be included dynamically. Clever!
(λx. x x) (λx. x x)

ClaudeM
Posts: 4
Joined: Fri Jun 21, 2019 11:46 pm

Re: Compile script to .exe with attached files

Post by ClaudeM »

Good evening,

I have successfully used the buildexe and exehead scripts on Linux, but failed on MS Windows. I am using newLISP 10.7.5 on both platforms.

What am I missing?

Thank you.

--
ClaudeM

ralph.ronnquist
Posts: 228
Joined: Mon Jun 02, 2014 1:40 am
Location: Melbourne, Australia

Re: Compile script to .exe with attached files

Post by ralph.ronnquist »

Well, as always, the first thing to exclude is "operator error". I don't know what the prevalent mistakes might be for Windows, but at least you should confirm that the program runs without being packed into a single executable. I.e., if the program is in the scripts A.lsp, B.lsp and C.lsp, with A.lsp being the main script, you would obviously test that by running

Code: Select all

newlisp A.lsp
Following that, you'll need to offer some more information. E.g. does it fail to build or does it fail to run a built .exe?

ClaudeM
Posts: 4
Joined: Fri Jun 21, 2019 11:46 pm

Re: Compile script to .exe with attached files

Post by ClaudeM »

It took a long time for me to get back to this. The actual problem was me getting the file order backward. I was fooled when I concatenated the two files into a single file with the main last and it worked. The main goes first.

I did some digging around - use vim to look at the executable and read the source code; it was an interesting journey. It was educational : converting a list to a string, etc.

Oddly, the main is just a string with embedded newlines, while the CSV library is text (with [text]; and [/text] delimiters).

newLISP is great!

Locked