visualdatatools.com

Bytesmiths · Posted: Tue Jan 17, 2012 11:21 pm Post subject: Taking output of a process?

I want to plot from MySQL, as well as from other UNIX processes.

Other than running a query to a file, then using that file from the dgraph CLI, is there any way to do this?

My UNIX orientation would like to simply pipe data into dgraph, something like "mysql -B -e 'SELECT * FROM data" | dgraph -script xxx.dgraph -output xxx.pdf", but that doesn't seem to work.

David · Posted: Wed Jan 18, 2012 2:57 pm Post subject:

I've thought about something like this. Is the standard in basically a tab delimited table?

How about adding a flag like -stdin to tell it to take in the content?
Maybe -stdout=pdf to tell it to stream the output as well.

David

Bytesmiths · Posted: Wed Jan 18, 2012 6:49 pm Post subject:

Bytesmiths · Posted: Wed Jan 18, 2012 6:55 pm Post subject:

PS: I'm happy to beta test any stdio comms you come up with..

David · Posted: Wed Jan 18, 2012 7:44 pm Post subject:

I'm going to think about if there are any side effects to treating the absence of a file as stdin. The -script flag is now optional, so dgraph foo.dgraph is the same as dgraph -script foo.dgraph. And even the output flag is optional since I can't import graphic files. It might be useful for some to be able to do a batch conversion using dgraph foo.dgraph out.pdf. Im not sure if that means that dgraph just hangs waiting for input or if the input is empty.

What would make sense for output flag. Maybe the absence of any output should mean stream as pdf. And then -output=png etc would specify what goes to standard out.

Regarding chunks. dgraph is really a small utility that calls the DataGraph application with arguments. So to do this I would save files into the /tmp directory which DataGraph that uses. There is also an additional twist coming soon which is the Sandbox requirement for all App store apps. The sandbox will restrict what applications can do, and limit files that can be read and written. So actually files have to be copied (possibly a hard/soft link will do) into a place which is freely readable. The entire home directory is basically blocked unless the user explicitly allows access through the Open/Save dialog. Since the command line isn't using that, you can't have sandboxed command line file filters (yet).

In fact, the dgraph command line utility might actually not be allowed inside the App wrapper, and I would move it to a "Install dgraph" menu entry that starts by downloading the dgraph app from my web site.

The good thing is that file access is a lot faster than in the old days, and file caching will mean that even if I write a file and then immediately DataGraph reads it, the access should be pretty fast.

David

Bytesmiths · Posted: Wed Jan 18, 2012 7:59 pm Post subject:

I don't envy the constraints you are under!

I haven't seen any documentation that the "-script" arg name is optional. Perhaps things would be simpler if it were required? Then an unadorned argument could always be interpreted as the name of a data file -- and its absence, the switch to stdin.

I like your idea of using a flag to specify the type of output. But "-output" wouldn't be simple to implement, if you're using the lack of "-output" to indicate "write to stdout", would it?

Perhaps a better way is to implement a new flag, such as "-outtype" that specifies the type of the output file. This would also be more "UNIXy" than re-using an existing flag for a different purpose.

You might also consider using the GNU arg processing functions. They make all this stuff a lot easier! Today, they're down (as are my websites) in solidarity with WikiPedia, but you might check these out tomorrow:

http://www.gnu.org/s/libc/manual/html_node/Argp.html

http://www.gnu.org/s/libc/manual/html_node/Getopt.html