C Api difficulty

[Jeff]

I'm having some difficulty with the following prototypes:

Code: Select all

memcached_return
memcached_mget (memcached_st *ptr, 
                char **keys,
                size_t *key_length, 
                unsigned int number_of_keys);

char *memcached_fetch (memcached_st *ptr,
                       char *key,
                       size_t *key_length, 
                       size_t *value_length,
                       uint32_t *flags, 
                       memcached_return *error);

I have MEMCACHED, in which I'm holding the pointer to memcached_st. I have MEMCACHED_RETURN, which I use to store the return pointer from various functions.

memcached_mget asynchronously gets the passed keys' values, and memcached_fetch retrieves them one by one. However, when watching the server debug output, when memcached_fetch is called, it's reporting an error (either SERVER_END or PROTOCOL_ERROR, and I can find documentation for neither). However, I think that the problem is newlisp dies with a bus error and disconnects in the middle of a transaction.

Here is the function I'm using (it's not yet cleaned up):

Code: Select all

(define (get-keys list-keys , res)
  (when MEMCACHED
    (letn ((num-keys (length list-keys))
           (keys (pack (trim (dup "lu " num-keys)) (map 'string list-keys)))
           (lengths (pack (trim (dup "d " num-keys)) (map 'length (map 'string list-keys)))))
      (setq MEMCACHED_RETURN (memcached_mget MEMCACHED keys lengths num-keys))
      (when (= (result) "SUCCESS")
        (setq res '())
        (dolist (key list-keys)
          (let ((key-length (length key))
                (val-length 0)
                (flags 0)
                (val nil))
            (setq val (memcached_fetch MEMCACHED
                                       key (address key-length)
                                       (address val-length) (address flags)
                                       (address MEMCACHED_RETURN)))
            (push (list key (get-string val)) res -1))))))
  res)

At the end, that get-string is where the bus error happens. memcached_fetch is returning 0 (NULL), which, according to libmemcached's docs, means I've retrieved all of the values... but it does this on the first call. Not quite sure where to go from here; the docs are not clear and I'm no expert in C.

Note: that 'result' function reads the error/status of MEMCACHED_RETURN. Success returns the string "SUCCESS", which suggests that the memcached_mget call succeeded.

Any ideas?

PS docs for libmemcached: http://docs.tangent.org/libmemcached/memcached.html

[Lutz]

Here are some comments:

(1) char * * keys, array of string pointers

The string (array) addresses have to be anchored to a symbol, so the strings don't get moved around (memory managed), but stay in a fixed place in memory.

Instead of:

Code: Select all

(set 'keys (pack (trim (dup "lu " num-keys)) (map string list-keys)))

do:

Code: Select all

(set 's-keys (map string list-keys))
(set 'keys (pack (dup "lu" num-keys) s-keys)

Here is a simple example to illustrate the same:

Code: Select all

(set 'sa '("ABC" "DEF" "GHI"))
(set 'ptr (pack "lululu" sa)) ; ptr -> char * *, ptr to array of ptrs
> (get-string (get-int (+ (address ptr) 0)))
"ABC"
> (get-string (get-int (+ (address ptr) 4)))
"DEF"
> (get-string (get-int (+ (address ptr) 8)))
"GHI"

(2) size_t *key_length

size_t should be 32-bit not 16-bit. And as in (1) you have to anchor addresses in memory to build an array of size_t unsigned integers.

Instead of:

Code: Select all

(set 'lengths (pack (trim (dup "d " num-keys)) (map 'length (map 'string list-keys))))

do:

Code: Select all

(set 'len-s-keys (map length (map string list-keys)))
; or using the previous s-keys
(set 'len-s-keys (map length s-keys))
(set 'lengths (pack (dup "lu" num-keys) len-s-keys))

Simple explicit example:

Code: Select all

(set 'sa '("A" "BC" "DEF"))
(set 'ptr (pack "lululu" sa)) 

(set 'l-sa (map length sa))
(set 'lengths (pack (dup "lu" 3) l-sa))

> (get-int (+ (address lengths) 0))
1
> (get-int (+ (address lengths) 4))
2
> (get-int (+ (address lengths) 8))
3

So now you can do:

Code: Select all

(setq MEMCACHED_RETURN (memcached_mget MEMCACHED keys lengths num-keys)) 

Hope this helps ;-)

[Jeff]

Is let-binding not sufficient? When I write the function with symbols locally bound in the variable and your tips:

Code: Select all

(define (fetch key , klen vlen flags res)
  (setq klen (address (length key))
        vlen (address 0)
        flags (address 0))
  (setq res (memcached_fetch MEMCACHED key klen vlen flags (address MEMCACHED_RETURN)))
  (if (= 0 res) nil (get-string res)))

(define (get-keys list-keys , res num-keys s-keys keys len-s-keys lengths)
  (setq MEMCACHED_RETURN nil)
  (when MEMCACHED
    (setq num-keys (length list-keys))
    (setq s-keys (map string list-keys))
    (setq keys (pack (dup "lu" num-keys) s-keys))
    (setq len-s-keys (map length s-keys)) 
    (setq lengths (pack (dup "lu" num-keys) len-s-keys)) 
    (setq MEMCACHED_RETURN (memcached_mget MEMCACHED keys lengths num-keys))
    (when (= (result) "SUCCESS")
      (setq res '())
      (dolist (key list-keys)
        (push (list key (fetch key)) res -1))))
  res)

...it works perfectly (and thanks!). Don't let-bound variables have their own addresses, or did I simply not bind enough of the intermediate variables to symbols?

[Lutz]

Is let-binding not sufficient?

Yes, absolutely sufficient, I was just to lazy to write out the whole 'let' -form and used 'set' for that reason, 'let' should work just the same.

[Jeff]

I've posted the new code to artful code (static.artfulcode.net/modules). Thanks for your help, Lutz.