Program to count lines of code (with filtering)

For the Compleat Fan
Locked
itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

Program to count lines of code (with filtering)

Post by itistoday »

Hey, I wrote a program to count lines of code.. it'll go down an entire directory tree. Move it to /usr/local/bin/numlines or wherever your preferred PATH is and run it like this:

$ cd path/to/project/dir
$ numlines

If you want to filter out someone else's code (say you have code from the Growl framework) you run it like this:

$ numlines Growl

If "Growl" is in the filename (case-sensitive) then it won't be counted as part of the total.

Code: Select all

#!/usr/bin/newlisp

; numlines

(constant 'S_IFLNK 40960)
(constant 'SIGINT 2)
(constant 'extensions '(".h" ".cpp" ".c" ".cc" ".m" ".java" ".lsp" ".lisp" ".sh" ".py" ".rb"))

(set 'total 0)

(define (string-contains L str)
	(!= '() (filter (fn (x) (find x str)) L))
)

(define (string-ends-with L str)
	(!= '() (filter (fn (x) (ends-with str x)) L))
)

(define (ctrlC-handler)
	(println "Total lines so far: " total)
	(exit 1)
)

(define (num-lines-in-file file , (num 0) fd)
	(if (string-ends-with extensions file)
		(if (string-contains (2 (main-args)) file)
			(if (> (length (2 (main-args))) 0)
				(println "skipping file: " file)
			)
			(begin
				(set 'fd (open file "r"))
				(if fd
					(begin
						(while (read-line fd) (inc 'num))
						(close fd)
						(println num "\t\tlines in: " file)
					)
					(println "*** couldn't open: " file)
				)
			)
		)
	)
	(inc 'total num)
)

(define (num-lines-in-dir dir)
	(change-dir dir)
	(dolist (e (directory))
		(if (!= "." (e 0))
			(if (directory? e)
				(if (= 0 (& S_IFLNK (file-info e 1)))
					(num-lines-in-dir e)
					(println "skipping symbolic link: " e)
				)
				(num-lines-in-file e)
			)
		)
	)
	(change-dir "..")
)

; begin program

(signal SIGINT 'ctrlC-handler)
(num-lines-in-dir ".")
(println "Total: " total " lines")
(exit)
Get your Objective newLISP groove on.

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Post by cormullion »

Nice work! I think I'll steal the code for detecting symlinks... :) (It doesn't follow MacOS aliases but they're not used much in this context.)

Only ~83,000 lines of newLISP here. Must work harder.

itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

Post by itistoday »

cormullion wrote:Nice work! I think I'll steal the code for detecting symlinks... :) (It doesn't follow MacOS aliases but they're not used much in this context.)

Only ~83,000 lines of newLISP here. Must work harder.
Wow, that's a lot of newLISP code! Whatcha workin' on? :)
Get your Objective newLISP groove on.

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Post by cormullion »

itistoday wrote:Wow, that's a lot of newLISP code! Whatcha workin' on? :)
:) Sadly nothing big or amazing. Just stuff I've started/finished/abandoned; OpenGL tutorials, maze makers and solvers, sources for blog posts, tokenizers, ASCII-art fonts (see Jack the Glypher), astronomical calculations, irc clients, two blogging apps, lots of half-finished guiserver projects. Perhaps the largest single file is the Markdown port (662 lines).

I tweaked your code a bit to produce a sorted list - then I thought that it would be even better to count the number of parentheses as well...! After all, there's nothing significant about a 'line' in newLISP source - perhaps a paren count indicates code structure (function density) more precisely? Also, it mostly ignores comments and fancy formatting.

Kazimir Majorinc
Posts: 388
Joined: Thu May 08, 2008 1:24 am
Location: Croatia
Contact:

Post by Kazimir Majorinc »

The program is really cute. Guys, if you are on the job, can you add counting of the tokens - parentheses included, one pair counted as two tokens? I'd like to have it as well, but improving that program is over my Newlisp head.

Cormullion, 83 000 of lines is very impressive, especially in Newlisp.

Kazimir Majorinc
Posts: 388
Joined: Thu May 08, 2008 1:24 am
Location: Croatia
Contact:

Post by Kazimir Majorinc »

On the second thought maybe it would be good to count parentheses, quotes (') and other tokens independently.

Kazimir Majorinc
Posts: 388
Joined: Thu May 08, 2008 1:24 am
Location: Croatia
Contact:

Post by Kazimir Majorinc »

On the second thought maybe it could be good to count parentheses, quotes (') and other tokens independently.

itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

Post by itistoday »

cormullion wrote:
itistoday wrote:Wow, that's a lot of newLISP code! Whatcha workin' on? :)
:) Sadly nothing big or amazing. Just stuff I've started/finished/abandoned; OpenGL tutorials, maze makers and solvers, sources for blog posts, tokenizers, ASCII-art fonts (see Jack the Glypher), astronomical calculations, irc clients, two blogging apps, lots of half-finished guiserver projects. Perhaps the largest single file is the Markdown port (662 lines).

I tweaked your code a bit to produce a sorted list - then I thought that it would be even better to count the number of parentheses as well...! After all, there's nothing significant about a 'line' in newLISP source - perhaps a paren count indicates code structure (function density) more precisely? Also, it mostly ignores comments and fancy formatting.
Would you like to post your version here? I'd be interested in see it. :-)
Get your Objective newLISP groove on.

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Post by cormullion »

Code: Select all

#!/usr/bin/env newlisp 

; numlines 

(constant 'S_IFLNK 40960) 
(constant 'SIGINT 2) 
(constant 'extensions '( ".lsp" )) 

(define (string-contains L str) 
  (!= '() (filter (fn (x) (find x str)) L)) 
) 

(define (string-ends-with L str) 
  (!= '() (filter (fn (x) (ends-with str x)) L)) 
) 

(define (ctrlC-handler) 
   (exit 1) 
)

(define (num-lines-in-file file , (num 0) (num-parens 0) fd) 
  (if (string-ends-with extensions file) 
   (if (string-contains (2 (main-args)) file) 
     (if (> (length (2 (main-args))) 0) 
      (print "skipping file: " file)
     ) 
     (begin 
      (set 'fd (open file "r")) 
      (if fd 
        (begin 
         (while (read-line fd) 
           (inc 'num)
           (replace {\(|\)} (current-line) $1 0)
           (inc 'num-parens $0)
           ) 
         (close fd) 
         (push (list num num-parens file) results) 
        ) 
        (println "*** couldn't open: " file) 
      ) 
     ) 
   ) 
  ) 
) 

(define (num-lines-in-dir dir)
  (change-dir dir) 
  (dolist (e (directory "./" {^[^.]}))
     (if (directory? e) 
      (if (= 0 (& S_IFLNK (file-info e 1))) 
        (num-lines-in-dir e) 
        (println "skipping symbolic link: " e) 
      ) 
      (num-lines-in-file e) 
     ) 
  ) 
  (change-dir "..") 
) 

; begin program 

(signal SIGINT 'ctrlC-handler) 
(num-lines-in-dir (string (env "HOME") "/lisp"))
(println (sort results (fn (x y) (< (x 1) (y 1)))))
(println "total lines " (apply add (map (fn (x) (x 0)) results)))
(println "total parentheses " (apply add (map (fn (x) (x 1)) results)))
(exit)

I just inserted a 'push' instead of a println.

Kasimir - your blog posts are excellent! I doubt whether these scripts are really very challenging...!

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Post by cormullion »

- of course that script is Unix-specific, I think, with the references to "..". Perhaps there should be a cross-platform version...

itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

Post by itistoday »

cormullion wrote:I just inserted a 'push' instead of a println.
Cool. :-)
Kasimir - your blog posts are excellent! I doubt whether these scripts are really very challenging...!
I agree, I especially enjoyed reading this one: Does the Function Really Evaluate to Itself?
Get your Objective newLISP groove on.

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Post by cormullion »

I wonder whether this take avoids a few of the cross-platform issues... Don't know, really...

Code: Select all

(constant 'extensions '(".lsp" "txt")) 

(define (string-ends-with L str) 
  (exists (fn (x) (ends-with str x)) L))

(define (count-in-file pn)
 (let (contents ""  returns 0 parens 0)
   (when  (string-ends-with extensions pn)
     (set 'contents (read-file pn)) 
     (replace "\n" contents (inc 'returns) 0)
     (replace {\(|\)} contents (inc 'parens) 0)
     (list returns parens pn))))
 
(define (count-in-tree dir)
   (dolist (nde (directory dir "^[a-z]"))  ; -----------?
     (set 'item (append dir "/" nde))
     (if (directory? item)
       (count-in-tree item)
       (if (set 'r (count-in-file item)) (push r results)))))

(count-in-tree (string (env "HOME") "/lisp"))

(when results 
  (sort results (fn (x y) (< (x 1) (y 1))))
  (println "total lines " (apply add (map (fn (x) (x 0)) results)))
  (println "total parentheses " (apply add (map (fn (x) (x 1)) results))))

(exit)

itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

Post by itistoday »

Hmm.. that's a nice way of doing it! Although I see two problems offhand with that script: first the directory separator for the system needs to be obtained somehow (I'm not sure if newLISP has a function to do that), because on windows it's "\" not "/", and secondly, if you're going to use the (directory) function in that way, you should probably regex on {^[^\.]} or something like that since folders can start with numbers, capital letters, etc.
Get your Objective newLISP groove on.

cormullion
Posts: 2038
Joined: Tue Nov 29, 2005 8:28 pm
Location: latiitude 50N longitude 3W
Contact:

Post by cormullion »

yes - you're right - A directory wildcard could possibly be cross-platform...

(I've never used Windows for scripting, so I wouldn't know.)

But - I seem to recall vaguely that Windows can use either "\" or "/"... ?

(Odd that two Mac users are fretting about Windows conventions.... :))

itistoday
Posts: 429
Joined: Sun Dec 02, 2007 5:10 pm
Contact:

Post by itistoday »

Hmm, yeah I don't know if windows can do "/" as well...
cormullion wrote:(Odd that two Mac users are fretting about Windows conventions.... :))
Haha. :-)
Get your Objective newLISP groove on.

Locked