Problems with time (now) Linux and Windows

Machine-specific discussion
Unix, Linux, OS X, OS/2, Windows, ..?
Locked
ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

Problems with time (now) Linux and Windows

Post by ale870 »

Hello,

I just discovered a problematic behaviour in newLisp.

I installled newLisp on a Linux and on a Windows computer.

If I use the function (now) on Linux it supply the correct offset values:

Code: Select all

> (now)
(2009 8 6 15 32 0 826945 218 5 -120 0)
Instead on Windows I receive a strange result:

Code: Select all

> (now)
(2009 8 6 15 26 39 844033 218 5 2089937716 3473408)
Please note that Windows and Linux time works well (show the correct date/time).

Where is the problem in Windows environment?

Thank you!
--

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

Unfortunately this has been broken for a long time on Win32 or working unreliable (at least since v9.3). Not sure when this will be fixed.

ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

Post by ale870 »

But the is related to Windows API or to newLisp implementation?
--

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

I just made it work. It will be in the next development version. It will be as described in the manual.

ale870
Posts: 297
Joined: Mon Nov 26, 2007 8:01 pm
Location: Italy

Post by ale870 »

:-) You are my hero Lutz (I'm joking!)

This fix is very important even for FOOP-Time.lsp module created by Cormullion, since that module does NOT work on Windows for this problem.

Thank you again!
--

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Post by cormullion »

Thanks Alessandro - I didn't realise that (now) didn't work. timeutilies depends on (now) working...

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

It's the last two fields: "minutes west of GMT" and "daylight savings time", which didn't work on many OSs. On Mac OS X at least the minutes-west field was working correctly.

In version 10.1.3, to be released this weekend, both fields are working on: Mac OS X, Win32, SunOS and Ubuntu LINUX.

On FreeBSD and OpenBSD, at least on the machines, I have access to, 0 is returned for both fields, which may be caused by a missing time-zone configuration on this machines (e.g. FreeBSD on nfshost and OpenBSD on m64 (Ted Walther?!)

Machines need to be supplied with a time-zone database containing data for different geographical areas in the world. If this is missing, 0 is returned on both fields.

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Post by TedWalther »

Lutz wrote:It's the last two fields: "minutes west of GMT" and "daylight savings time", which didn't work on many OSs. On Mac OS X at least the minutes-west field was working correctly.

In version 10.1.3, to be released this weekend, both fields are working on: Mac OS X, Win32, SunOS and Ubuntu LINUX.

On FreeBSD and OpenBSD, at least on the machines, I have access to, 0 is returned for both fields, which may be caused by a missing time-zone configuration on this machines (e.g. FreeBSD on nfshost and OpenBSD on m64 (Ted Walther?!)

Machines need to be supplied with a time-zone database containing data for different geographical areas in the world. If this is missing, 0 is returned on both fields.
I checked, timezone is properly configured (look at /etc/localtime, it is a link to a file in /usr/share/zoneinfo, which is a precompiled timezone file) Lots of info in the "zic" manpage, which stands for zone-info-compiler.

From the manpage of gettimeofday():
int gettimeofday(struct timeval *tv, struct timezone *tz);

The use of the timezone structure is obsolete; the tz argument should normally be specified as NULL. The tz_dsttime field has never been used under Linux; it has not been and will not be sup‐
ported by libc or glibc. Each and every occurrence of this field in the kernel source (other than the declaration) is a bug. Thus, the following is purely of historic interest.
May I recommend removing the last two fields from being returned by (now)?

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Post by cormullion »

TedWalther wrote:May I recommend removing the last two fields from being returned by (now)?
Or rather, by documenting their problems? Otherwise code would break with 'list out of bounds'.

Lutz
Posts: 5289
Joined: Thu Sep 26, 2002 4:45 pm
Location: Pasadena, California
Contact:

Post by Lutz »

The only place where I have used 'struct timezone' was for the _BSD flavor and after checking this documentation for time.h:

http://fxr.watson.org/

For Mac OS X, UBUNTU Linux and SunOS I am using the GNU 'C' predefined variables 'int timezone' and 'int daylight'. But they don't work on FreeBSD and OpenBSD. For Mac OSX 'struct timezone' works too, but I wen with 'int timezone' and 'int daylight', as hey are more common.

We have it working now on Mac OS X, UBUNTU Linux, SunOS and Win32, so I will not drop it just because it doesn't work FreeBSD/OpenBSD ;-).



Ps: also found tm_gmtoff, and ltm->tm_isdst in 'struct tm' on BSDs, but they don't work either.

There must be a way to do this correctly on the BSD's, can somebody help? Specially FreeBSD is important as it is used so frequently at ISPs.

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Post by TedWalther »

Lutz, are you calling tzset to populate the extern char *tzname[2]; variable?

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Post by TedWalther »

I just checked; the gmtime() and localtime() calls on OpenBSD 4.4 do return the correct data.

The tm_isdst field is an integer that indexes the tzname external variable, which contains the name of the timezone. Here is the output of a test program I wrote:

Code: Select all

tzname[0]: PST
tzname[1]: PDT
tm.tm_sec: 15
tm.tm_min: 29
tm.tm_hour: 16
tm.tm_mday: 8
tm.tm_mon: 7
tm.tm_year: 109
tm.tm_wday: 6
tm.tm_yday: 219
tm.tm_isdst: 1
tm.tm_gmtoff: -25200
tm.tm_zone: PDT
With the gmtime() function I would expect isdst and gmtoff to always be 0. For localtime, you must run tzset() first.

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Post by TedWalther »

Here is the source of the test program, perhaps you will find it useful to run it on the other platforms as well, perhaps the Mac OSX and SunOS platforms.

Code: Select all

#include <sys>
#include <time>
#include <stdio>

extern char *tzname[2];

int
main(int argc, char** argv) {
        time_t t;
        struct tm *st;

        tzset(); // this populates tzname
        t = time(NULL);
        st = localtime(&t);
        printf("tzname[0]: %s\n", tzname[0]);
        printf("tzname[1]: %s\n", tzname[1]);
        printf("tm.tm_sec: %d\n", st->tm_sec);
        printf("tm.tm_min: %d\n", st->tm_min);
        printf("tm.tm_hour: %d\n", st->tm_hour);
        printf("tm.tm_mday: %d\n", st->tm_mday);
        printf("tm.tm_mon: %d\n", st->tm_mon);
        printf("tm.tm_year: %d\n", st->tm_year);
        printf("tm.tm_wday: %d\n", st->tm_wday);
        printf("tm.tm_yday: %d\n", st->tm_yday);
        printf("tm.tm_isdst: %d\n", st->tm_isdst);
        printf("tm.tm_gmtoff: %d\n", st->tm_gmtoff);
        printf("tm.tm_zone: %s\n", st->tm_zone);
}

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Post by TedWalther »

Interesting, according to the GNU website, it is better to rely on the BSD method than on the extern timezone variable:

http://www.gnu.org/s/libc/manual/html_n ... -Functions
In GNU programs it is better to use tm_gmtoff, since it contains the correct offset even when it is not the latest one.
I just tested the BSD method on Ubuntu, and it does indeed work. The essential 3 lines would be:

Code: Select all

tzset();
t=time(NULL);
st=localtime(&t);
Except for Win32, I believe this should work on all Unixes from the past 10 years. And maybe even on Win32!

Ted

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Post by TedWalther »

the problem is somehow in how stuffIntegerList() is being called. Or maybe stuffIntegerList. Because the call to localtime() is returning the correct info. I verified this, after putting a call to tzset() in the main() function of newlisp. Then I put in some printf debugging, verified that the correct values are coming out of localtime(). Once they are put into stuffIntegerList, what comes out the other end of the newLISP prompt is 0 and 0.

Is it a problem that tm_gmtoff is negative, and not positive? the (UINT) cast looks possibly suspicious to me, although it didn't prevent printf() from accurately reporting the negative value when using the %d specifier.

Ted

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Post by TedWalther »

I found the problem. gmtime() and localtime() (and a couple other functions) all put their result into the same underlying structure, and they return a pointer to it. So when you are calling gmtime(), Lutz, you are blasting over the values put there by localtime(). I'll have a patch ready for you shortly.

By the way, that behavior is documented in the BSD manpage and is allowed by POSIX.

Ted

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Problem solved!

Post by TedWalther »

Lutz, why are you using gettimeofday() instead of time()?

The fixed files newlisp.c and nl-filesys.c have been mailed to you. Using the (corrected) BSD method for everything except WIN_32 saves 9 lines of code, although 5 of those lines were comments. It shouldn't change the behavior for any platform.

On BSD, it is necessary to call tzset() before calling localtime(), and can't do any harm on other POSIX platforms.

Ted

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

Post by TedWalther »

I did another test. The call to tzset() may not even be necessary. I'll see make an even smaller patch; if that works on all platforms, then the bigger patch can be tossed. I just wanted to be extra safe.

[some time later...]

Although it seems to work without tzset(), tzset() should be called for another reason; using tzset() makes calls to localtime() utilize the environment variable TZ, allowing users to set the timezone on the fly in the traditional Unix method.

The new, less intrusive patch reduces the lines of code by 12, and I suspect I could cut out another couple lines of code; do AIX, TRU64, and OS2 all not support tzset(), gettimeofday(), localtime(), and gmtime()?

Ted

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

The latest patch:

Post by TedWalther »

Code: Select all

--- n1/newlisp.c        Sat Aug  8 12:15:17 2009
+++ n3/newlisp.c        Sat Aug  8 21:33:39 2009
@@ -593,6 +593,7 @@
 #ifndef OS2
 pagesize = getpagesize();
 #endif
+tzset();
 #endif
 
 initLocale();
diff -ru n1/nl-filesys.c n3/nl-filesys.c
--- n1/nl-filesys.c     Sat Aug  8 12:15:17 2009
+++ n3/nl-filesys.c     Sat Aug  8 21:28:03 2009
@@ -2315,29 +2315,26 @@
 {
 struct timeval tv;
 struct tm *ttm;
-#ifdef _BSD
-struct tm *ltm;
-#endif
 #ifdef WIN_32
 TIME_ZONE_INFORMATION timeZone;
+#else
+UINT _gmt_offset;
+UINT _dst_predicate;
 #endif
-/* struct timezone tzp; obsolete? */
-ssize_t offset = 0;
 
 gettimeofday(&tv, NULL);
 
 if(params != nilCell)
        {
        getInteger(params, NULL);
-       offset *= 60;
-        tv.tv_sec += offset;
        }
 
-#ifdef _BSD
-ltm = localtime((time_t *)&tv.tv_sec);
-#endif
 #ifdef WIN_32
 GetTimeZoneInformation(&timeZone);
+#else
+ttm = localtime((time_t *)&tv.tv_sec);
+_gmt_offset = (UINT)ttm->tm_gmtoff;
+_dst_predicate = (UINT)ttm->tm_isdst;
 #endif
 
 ttm = gmtime((time_t *)&tv.tv_sec);
@@ -2353,28 +2350,18 @@
     (UINT)tv.tv_usec,
     (UINT)ttm->tm_yday + 1,
     (UINT)ttm->tm_wday + 1,
-
-#if defined(MAC_OSX) || defined(LINUX) || defined(SUNOS)
-    (UINT)timezone / 60, (UINT)daylight
-#endif
-#if defined(_BSD) 
-/* this is as of BSD docs but will work only Mac OS X.
-   FreeBSD and OpenBSD give 0, 0
-    (long)tzp.tz_minuteswest, (long)tzp.tz_dsttime
-   and the following doesn't work either, -> 0, 0
-*/
-    (UINT)ltm->tm_gmtoff, (UINT)ltm->tm_isdst
-#endif
 #if defined(WIN_32)
      timeZone.Bias,
      timeZone.DaylightBias     
-#endif
-#if defined(OS2) || defined(TRU64) || defined(AIX)
+#elif defined(OS2) || defined(TRU64) || defined(AIX)
 #ifdef NEWLISP64
     (UINT)0L, (UINT)0L
 #else
     (UINT)0, (UINT)0
 #endif
+#else
+     _gmt_offset,
+     _dst_predicate
 #endif
     ));
 }

TedWalther
Posts: 608
Joined: Mon Feb 05, 2007 1:04 am
Location: Abbotsford, BC
Contact:

updated patch

Post by TedWalther »

I got the code wrong; I hadn't looked closely at what value the offset was supposed to hold; tm_gmtoff holds seconds east of GMT, rather than the minutes west of GMT we are trying to return. Here is the fix:

Code: Select all

diff -ru n3/nl-filesys.c n4/nl-filesys.c
--- n3/nl-filesys.c     Sat Aug  8 21:41:03 2009
+++ n4/nl-filesys.c     Sat Aug  8 22:24:39 2009
@@ -2333,7 +2333,7 @@
 GetTimeZoneInformation(&timeZone);
 #else
 ttm = localtime((time_t *)&tv.tv_sec);
-_gmt_offset = (UINT)ttm->tm_gmtoff;
+_gmt_offset = (UINT) (((ttm->tm_gmtoff > 0) ? 86400 - ttm->tm_gmtoff : 0 - ttm->tm_gmtoff) / 60);
 _dst_predicate = (UINT)ttm->tm_isdst;
 #endif
 

Locked