Page 1 of 1
mingw64 build patch
Posted: Sat Feb 14, 2015 7:16 pm
by kosh
Hi Lutz.
I wrote patch file for builds with MinGW-w64 gcc.
Changes are following:
- Define UINT as pointer size integer (stdint.h, uintptr_t)
- Use type conversion macros for printf (inttypes.h, PRIdPTR)
- Auto detect 64-bit memory model (LP64/LLP64) using C pre-defined macros
- nl-sock.c: bugfix non initialized var DefaultIn
Patch files can be viewed on GitHub:
https://github.com/kosh04/newlisp/compa ... /mingw-w64
(p.s. BBS attachment fails with message
The extension diff is not allowed.)
Build and Test (`make testall`) passed under the following platform:
- Win7 Home amd64
-- TDM-GCC (mingw-w64 gcc 4.9.1)
-- MinGW gcc 4.8.1
-- cygwin32 gcc 4.9.2
-- cygwin64 gcc 4.9.2
- Mac OSX clang
However, this patch have some problems:
- stdint.h, inttypes.h headers available in ISO C99. but newLISP C source seem to C90.
- win32-path.c: an incomplete compatibility between `struct stat` and `struct _stat`.
- nl-sock.c: UINT type is name conflict in ws2tcpip.h.
This patch provides the temporary workaround using push_macro/pop_macro.
If you don't like this way, better to rename the UINT to another name (e.g. `typedef uintptr_t nl_int;`)
All reviews are welcome. Thanks.
Re: mingw64 build patch
Posted: Sat Feb 14, 2015 7:32 pm
by kosh
Re: mingw64 build patch
Posted: Sun Feb 15, 2015 4:17 pm
by Lutz
Thanks for this great work! Everything worked right away on all OS.
All integrated for 10.6.3 here:
http://www.newlisp.org/downloads/develo ... nprogress/
(several reposts after initial post)
Tested on: OS X 10.10, Ubuntu Linux 14.04, FreeBSD 9.3, OpenBSD 5.4 and Windows XP sp2 with mingw32.
I wonder what the speed comparison is between the 32-bit and 64-bit versions of newLISP on Windows.
Re: mingw64 build patch
Posted: Mon Feb 16, 2015 7:48 pm
by kosh
Thanks Lutz for merge patch.
I found a missing code in newlisp.h (newlisp-10.6.3.tgz)
Code: Select all
#undef NEWLISP64
#if defined(_LP64) || defined(_WIN64)
#define NEWLISP64
#endif
and qa-specific-tests/qa-local-domain (for `make testall`)
Code: Select all
diff --git a/qa-specific-tests/qa-local-domain b/qa-specific-tests/qa-local-domain
index caa5901..49b41f7 100755
--- a/qa-specific-tests/qa-local-domain
+++ b/qa-specific-tests/qa-local-domain
@@ -6,6 +6,10 @@
(println)
(println "Testing UNIX local domain sockets")
+(when (find ostype '("Win32" "OS/2"))
+ (println "not tested on " ostype)
+ (exit))
+
(define (listener path)
(set 'lsock (net-listen path))
(set 'csock (net-accept lsock))
NEWLISP64 is defined in files makefile_* in this time.
It will more better way to use predefined compiler macros.
Code: Select all
# check predefined macros
$ echo | gcc -m32 -dM -E - | grep WIN64
$ echo | gcc -m64 -dM -E - | grep WIN64
#define _WIN64 1
#define __WIN64 1
#define WIN64 1
#define __WIN64__ 1
Lutz wrote:I wonder what the speed comparison is between the 32-bit and 64-bit versions of newLISP on Windows.
FYI, my AppVeryor's build log info is the following (mingw cross-compile on cygwin):
https://ci.appveyor.com/project/kosh04/ ... ild/1.0.62
Code: Select all
# 32-bit
>>>>> total time: 14779.53999999999
>>>>> Performance ratio: 5.70 (1.0 on MacOSX 10.9, 2.3GHz Intel Core i5, newLISP v10.6.0-64-bit)
# 64-bit
>>>>> total time: 12015.73600000001
>>>>> Performance ratio: 4.64 (1.0 on MacOSX 10.9, 2.3GHz Intel Core i5, newLISP v10.6.0-64-bit)
My build environment info will be posted at a later.
Regards.
Re: mingw64 build patch
Posted: Tue Feb 17, 2015 8:34 pm
by Lutz
Thanks Kosh. The speed difference between Win32 and Win64 seems to be similar to other platforms.
Currently the selection between 32 and 64 bit is made by the configure script, which then copies one of the makefile_xxx to makefile_build. This gives people the possibility to decide for a different memory model by picking a different makefile_xxxx. This happens often when building for embedded or other resource constrained systems. So, at this point automatic NEWLISP64 detection will not happen yet, but may be added in the future.
The configure-alt script from Ted Walther works a bit different creating makefile_build by building it from scratch and doing more automation in discovering features on different OS platforms. Win64 capability has not yet been added, but that should be easy.
I added detection of Windows 64-bit to configure but currently don’t have the possibility to test it, as my Windows development platforms currently is only 32-bit. I hope the script and new makefiles work for you.
Many qa-specific-test/qa-xxx files have been changed testing for Win64, where appropriate. A few makefiles for mingw64 have been added and should be picked out automatically by the configure script. After fresh install of the source distribution or after a ‘make clean’ a simple ‘make’ will call the configure script first then make the newlisp executable.
Let me know if all this works when using mingw64 tools.
Ps: some more fixes uploaded at 2015-2-18 15:47 UTC
Re: mingw64 build patch
Posted: Wed Feb 18, 2015 8:43 pm
by kosh
makefile_mingwLLP64_utf8 is here:
Code: Select all
# makefile for newLISP v. 10.x.x on MinGW-w64 with UTF-8 support
OBJS = newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o nl-sock.o \
nl-import.o nl-xml-json.o nl-web.o nl-matrix.o nl-debug.o pcre.o nl-utf8.o win-util.o win-path.o
CFLAGS = -Wall -Wno-uninitialized -Wno-long-long -O1 -g
CPPFLAGS = -DWIN_64 -DSUPPORT_UTF8
#CPPFLAGS = -DWIN_32 -NEWLISP64 -DSUPPORT_UTF8
LDLIBS = -lws2_32
CC = gcc
STRIP = strip
default: newlisp.exe
newlisp.exe: $(OBJS)
$(CC) $(LDFLAGS) $^ $(LDLIBS) -o $@
$(STRIP) $@
newlisp.dll: LDFLAGS += -static-libgcc -Wl,--enable-stdcall-fixup
newlisp.dll: $(OBJS) win-dll.o win-dll.def
$(CC) -shared $(LDFLAGS) $^ $(LDLIBS) -o $@
$(OBJS): primes.h protos.h newlisp.h makefile_mingwLLP64_utf8
This makefile can also make DLL file:
Code: Select all
make -f makefile_mingwLLP64_utf8 newlisp.dll
---
Unfortunatly, new newlisp-10.6.3.tgz compile with errors:
Code: Select all
gcc -Wall -Wno-uninitialized -Wno-long-long -O1 -g -DWIN_64 -DSUPPORT_UTF8 -c -o nl-sock.o nl-sock.c
In file included from nl-sock.c:29:0:
C:/MinGW64/x86_64-w64-mingw32/include/ws2tcpip.h:160:1: error: size of unnamed array is negative
C_ASSERT(sizeof(IN_PKTINFO)==8);
^
C:/MinGW64/x86_64-w64-mingw32/include/ws2tcpip.h:167:1: error: size of unnamed array is negative
C_ASSERT(sizeof(IN6_PKTINFO)==20);
^
make: *** [nl-sock.o] Error 1
in nl-sock.c: `#define UINTP uintptr_t` is not ineffective. I recommend to use push_macro/pop_macro, or include newlisp.h after ws2tcpip.h if you don't rename the type name UINT.
---
I hope the value ostype be always "Win32".
This value should not be distinguished by 32/64 bit as well as in other platform. To determine the 64-bit platform can use sys-info.
Re: mingw64 build patch
Posted: Thu Feb 19, 2015 3:57 pm
by Lutz
Thanks Kosh, I will continue to be integrate Windows 64-bit development when I have a development environment similar to yours, with both mingw32 and mingw64 together and the latest stable release.
The logical would be to use a new ostype string “Windows”, although it would break backwards compatibility with code using “Win32”. Using both “Win32” and “Win64” would leave compatibility but be impractical when running old “Win32” scripts on “Win64” installations, although probably those scripts are compatible.
As soon, as I have a new development environment installed, a decision will be made.
Re: mingw64 build patch
Posted: Thu Mar 19, 2015 8:31 pm
by kosh
Lutz, you have completed the setup of new Windows machine?
As an aside,
http://tdm-gcc.tdragon.net/ provide "TDM32 Packages". so it works on 32-bit Windows XP.
Of course, a generated newlisp.exe runs on 64-bit Windows.
Re: mingw64 build patch
Posted: Thu Mar 19, 2015 11:52 pm
by Lutz
Not until April, I will have access to a Windows 7 64-bit machine, but thanks for the tip using TDM gcc. That will make the installation a lot easier. I understand that you can cross compile to 64-bint on a 32-bit machine, but I don't have enough space too install, so it has to wait.
Re: mingw64 build patch
Posted: Thu Apr 02, 2015 1:25 pm
by Lutz
A Windows 64-bit version made with tdm gcc 64 is now working, but only for IPv4. Perhaps IPv6 is not supported in the tdm version of mingw gcc? Hve not looked into it yet.
The following makefiles have been added: makefile_tdmgcc64 and makefile_tdmgcc64dll and the builds pass make testall except with qa-net6.
Other makefiles including UTF8 and extended FFI using libffi will follow during the next days.
Currently I am still using Win32 and Win64 for the newLISP variable ostype. May be this will change to Windows (64/32 info in in sys-info), as Kosh is suggesting. When the other 64-bit flavors are ready, a .exe will published too for people to test, who can not compile themselves.
Re: mingw64 build patch
Posted: Thu Apr 02, 2015 3:10 pm
by kosh
> but only for IPv4. Perhaps IPv6 is not supported in the tdm version of mingw gcc? Hve not looked into it yet.
Maybe, It because you use variable `defaultIn` without zero-initialization.
https://github.com/kosh04/newlisp/compa ... 1a5cbfL159
initDefaultInAddr() in nl-sock.c
Code: Select all
-defaultIn = allocMemory(defaultInLen);
+defaultIn = callocMemory(defaultInLen);
Re: mingw64 build patch
Posted: Thu Apr 02, 2015 4:06 pm
by Lutz
Re: mingw64 build patch
Posted: Thu Apr 02, 2015 4:18 pm
by Lutz
... and updated again.
Re: mingw64 build patch
Posted: Thu Apr 09, 2015 8:10 am
by Lutz
64-bit 10.6.3 newLISP executables with and without UTF8 suppport can be found here:
http://www.newlisp.org/downloads/development/inprogress
Both .exe are compiled on Windows 7 using mingw64 gcc.
Re: mingw64 build patch
Posted: Sat Apr 11, 2015 5:11 pm
by xytroxon
Hi Lutz!
This Win64 version is noticeably faster than the Win32 version - like 7 times faster for some string operations!!!
Then I noticed I was having some missing output using either newlisp64.exe or newlisp64utf8.exe.
Assuming an 80 character x 25 line console display - the following code should display 6000 chars in three sections of 2000 chars each of "+", "-" and "*". But the Win64 newlisp versions will display the first 2000 char "+" section, and of the second 2000 char "-" section, only about the first 50 chars are displayed, and nothing of the third 2000 char "*" section. Sometimes the end of the last line will also show 5 or 6 random chars from the 0 to 31 and 127 to 255 ranges of ascii characters - like a wild pointer running on into another memory variable.
The string length correctly shows all 6000 chars of the "test" string being present, and this code works fine with the last Win32 release of newlisp.
Code was tested on both "console2" and "conemu" Windows console emulator programs with simular results.
Code: Select all
(setq a (string (dup "+" (* 80 25))))
(setq b (string (dup "-" (* 80 25))))
(setq c (string (dup "*" (* 80 25))))
(setq test (string a b c))
(print (length test) "\n" test)
(exit)
-- xytroxon
Re: mingw64 build patch
Posted: Sat Apr 11, 2015 8:31 pm
by xytroxon
Doing a write-file of the "test" string shows no premature shortening of the string or any extra "odd" chars in the file on disk.
But reading and printing the file has the same display failure.
So maybe the problem lies within the print function.
--xytroxon
Re: mingw64 build patch
Posted: Sun Apr 12, 2015 5:15 am
by TedWalther
I remember a few years ago, Phillip Whelan was able to very quickly locate and patch a 64 bitness problem in newLisp. Wish he was still around, this sounds like a 64 bitness type of problem, right up his alley.
Re: mingw64 build patch
Posted: Sun Apr 12, 2015 7:08 am
by Lutz
Cannot repeat this problem. Your test code or printing any other string >= 6000 characters always works on Windows 7 Home Premium SP1 command shell cmd.exe or using the MSYS/MinGW bash shell. I wonder if other Windows 7 users can repeat this problem. What version of Windows are you using?
Re: mingw64 build patch
Posted: Sun Apr 12, 2015 8:43 am
by HPW
Hello,
I tried it on my win7 home premium tablet.
When I paste it line by line into the newlisp Shell it works as expected.
But when i make a file Test.lsp with:
Code: Select all
(setq a (string (dup "+" (* 80 25))))
(setq b (string (dup "-" (* 80 25))))
(setq c (string (dup "*" (* 80 25))))
(setq test (string a b c))
(silent(print (length test) "\n" test))
and do a (load "Test.lsp") in a fresh started Shell I get:
Code: Select all
newLISP v.10.6.3 64-bit on Windows IPv4/6 libffi, options: newlisp -h
> (load "Test.lsp")
6000
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
--------------------------------------------------------¦.6Å-ï?
This seems to be Buggy.
Edit: The 32 bit Exe does this correctly.
Regards
Hans-Peter
Re: mingw64 build patch
Posted: Sun Apr 12, 2015 11:19 am
by Lutz
Yes, now I can repeat it too when loading the test code from a file. I could not repeat it when testing the code interactively, but it also fails when interactively when using silent.
It also fails on the 32-bit version in 10.6.3. It seems that this has nothing to too with 32-bit versus 64-bit, but is related to other changes in 10.3.6 development. I am glad we are catching this before 10.6.3 goes out of "inprogress".
Re: mingw64 build patch
Posted: Sun Apr 12, 2015 1:02 pm
by kosh
I also confirmed this IO problem.
Code: Select all
$ newlisp64.exe -n
> (setf str (string (dup {X} 2049) {HELLO}))
[text]XXXX...<repeat 2049 times>HELLO[/text]
> (println str)
XXXX...<repeat 2049 times>
It seems that cause internal output function `my_vasprintf` (in nl-filesys.c, called from varPrintf)
`vasprintf` has been implemented in mingw64 in stdio.h (but mingw32 yet).
then, It could be fixed in this way:
Code: Select all
/* newlisp.h */
#ifdef WINDOWS
#ifndef __MINGW64__ /* macros depends on mingw implementation */
# define MY_VASPRINTF
# define vasprintf my_vasprintf
#endif
...
#endif
Regards.
Re: mingw64 build patch
Posted: Sun Apr 12, 2015 2:40 pm
by Lutz
-
Posted: Wed Jun 29, 2022 11:33 pm
by ValentiMr
Attached is a patch to fix the test build on head. All it does it make it so the tests compile: they still don't pass at least they don't pass on mac os x.
Also, I've noticed a bunch of build issues on Mac OS X. Anyone mind if I fix those?