visualdatatools.com Forum Index visualdatatools.com
Discussion for DataTank and DataGraph
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Snapshot / Toggle

 
Post new topic   Reply to topic    visualdatatools.com Forum Index -> Using External Programs
View previous topic :: View next topic  
Author Message
stschiff



Joined: 04 Mar 2009
Posts: 34
Location: Cologne, Germany

PostPosted: Fri Jan 15, 2010 1:37 pm    Post subject: Snapshot / Toggle Reply with quote

Hi,

I read in the help files that one can use a snapshot to save a result of a simulation run, e.g. from a task variable, into the script.
Can one do that with many runs for different parameters? How would DataTank help me organizing my files?

Here is what I do so far:
I run the simulation from the command line, the output is written to the standard out, which allows me to have several runs using command line arguments, for example:
mySim -mu 0.5 > myResult_mu0.5.dtascii
mySim -mu 2.5 > myResult_mu2.5.dtascii
mySim -mu 5 > myResult_mu5.dtascii
mySim -mu 7.5 > myResult_mu7.5.dtascii
and so on...

Then I go into dataTank and make a DataFile-Open module, and select the run that I want to look at.

Now, the snapshot module could maybe help me doing that... for that I would have let my program actually output a real file like output.dtascii, and then I run the task from within dataTank and save the result using a snapshot. Would that be possible with a whole bunch of runs like above?

Also, I guess the toggle module can only switch between two different files...

Another possibility I tried was the filename-folder content module. That one shows me all my results and maps them onto different time-values, however, the time-slider won't switch between the files, maybe because each of my files contains output-variables than themself depend on time...

Sorry for asking many questions at once, I appreciate comments on any one of the points Smile

Thanks,

Stephan
Back to top
View user's profile Send private message
David
Site Admin


Joined: 25 Nov 2006
Posts: 1966
Location: Chapel Hill, NC

PostPosted: Fri Jan 15, 2010 4:18 pm    Post subject: Reply with quote

There is a way to have time series of time series. This is what the Series object allows you to do.

However it is a little hard getting data into a Series object without using C++.

What type of structure is inside myResult_mu**.dtascii? (feel free to send them as attachment or point me to a download site). Do you want to run a test on each "mu" and then collect that information as a function of mu?

Also, do you want DataTank to launch the mySim runs for you?

David
Back to top
View user's profile Send private message Send e-mail
stschiff



Joined: 04 Mar 2009
Posts: 34
Location: Cologne, Germany

PostPosted: Sat Jan 16, 2010 5:26 pm    Post subject: Reply with quote

The point is, that my program can run quite long for certain parameters, and therefore I want to save the results. There is many different ways of achieving this with DataTank as I believe. I am trying to figure out which is the most elegant one. Here are possible solutions:

1.) I can just manually run the tasks from the command line and save the outputs for different parameters in several different files. In my post-processing scripts I can then look at them individually with the "Open" module. The disadvantage is that I have to manually go to the command line each time I run different parameters.

2.) I can set up a task or external program variable and run the program from within DataTank. This works, but how can I then save the result and have the flexibility to look at different runs? I thought the Snapshot/Toggle thing might help but I don't really get it, since that would only save one single run as I understand

3.) I tried out the "Parameter Run" module, works quite nice on my local machine. But the help file says I should then look at the results with a "from Parameter Run" module. This does not seem to exist, does it? So right now I don't know how to use the parameter run results. Also I don't know how to later extent this list of runs with even more runs for certain parameters, i.e. how flexible is it?


And finally, I want to be able to condense some of the results of each run into one single script. For example, each run outputs a time-sequence of numbers, I want to get the mean over time for each run and make a single plot with that mean dependent on the different parameters I used. So I guess I should make script that loads a single file and computes the mean, and than I somehow use this script externally to compute many different means. Maybe I make a small example dataset to make this clearer, as you offered to look at a more concrete example.

Thanks,

Stephan

P.S. C++ is not a problem at all, especially since I know that I can still import and link my ObjC-Frameworks.
Back to top
View user's profile Send private message
David
Site Admin


Joined: 25 Nov 2006
Posts: 1966
Location: Chapel Hill, NC

PostPosted: Sat Jan 16, 2010 7:34 pm    Post subject: Reply with quote

It would be helpful to get the data files.

The Series object would allow you to do this. Assume you have a parameter sweep where each parameter is a time sequence. Then the parameter would be the time, and the time sequence would be a single "Series" object. There is a module called "Script On Series" that will allow you to hand each time sequence into a separate DataTank script, and return back values. This can then be sampled.

If you send me data I can do this and build a sample project that I can link from the help page. The sample project would not depend on your particular data but instead have the same structure.

David
Back to top
View user's profile Send private message Send e-mail
stschiff



Joined: 04 Mar 2009
Posts: 34
Location: Cologne, Germany

PostPosted: Sun Jan 17, 2010 8:59 pm    Post subject: Reply with quote

OK, I sent you something via email.

A quick question: In normal datafiles it's easy to save time-dependent variables. I use this for example in my simulation, which I can run as a task-variable.

But if I use an external program module instead of the task module, and changing the result variable to "Var" of course, this does not seem to work. Only the first time-value is actually transmitted as the result into the script. Is this on purpose?

And related to that: Why can't I set up an external program that returns a Series-Variable? The Series-Variable does not have a computational part called "External Program"...

Thanks,
Stephan
Back to top
View user's profile Send private message
David
Site Admin


Joined: 25 Nov 2006
Posts: 1966
Location: Chapel Hill, NC

PostPosted: Mon Jan 18, 2010 12:37 am    Post subject: Reply with quote

I just made a DataTank project that shows this (Series of Series) and put it on the help site. This shows you how to use a series of a series. I updated DataTank a little bit to streamline things.

Regarding the specific questions.
There is not an External Program module for the series yet, but there should be. It needs to be a little bit different just like the external program module for the group. In the group entry you can create a series of variables, and I need to create a very similar setup for the series.

It is slightly tricky to understand this. The main issue is that DataTank handles a series of variables. The Series variable type can be a series of any variable (the icon adapts) so you can have a time series of a variable where every time value is a time series. The "Script on Series" module is a way to hand each time value (which is a time series) into a separate DataTank file which will look at that inner time series as the main time series. Note that if DataTank decides to compute a time series, then all time values of that time series are computed.

David
Back to top
View user's profile Send private message Send e-mail
stschiff



Joined: 04 Mar 2009
Posts: 34
Location: Cologne, Germany

PostPosted: Mon Jan 18, 2010 7:41 am    Post subject: Reply with quote

David wrote:
I just made a DataTank project that shows this (Series of Series) and put it on the help site. This shows you how to use a series of a series. I updated DataTank a little bit to streamline things.


Thanks a lot. These help examples are really cool. Especially with the inlined text!

David wrote:
Regarding the specific questions.
There is not an External Program module for the series yet, but there should be. It needs to be a little bit different just like the external program module for the group. In the group entry you can create a series of variables, and I need to create a very similar setup for the series.

It is slightly tricky to understand this. The main issue is that DataTank handles a series of variables. The Series variable type can be a series of any variable (the icon adapts) so you can have a time series of a variable where every time value is a time series. The "Script on Series" module is a way to hand each time value (which is a time series) into a separate DataTank file which will look at that inner time series as the main time series. Note that if DataTank decides to compute a time series, then all time values of that time series are computed.

David


Right. I understood this, I already played a bit around with Series. I think that your help document, which explains this as a loop structure, is good. I also successfully created an external program that creates a group. The structure is quite clear. However, there seems to be a little bug, related to the fact that there is no external program module for a series:
If I add a Series Variable to the Group, in DataTank, it complains that the Series has no type, and there is no way of indicating the type. (I am talking about adding a Series variable to the DT_RetGroup in an External Program Module in Datatank, before actually looking at XCode).
Is this also the reason that your example external program creates a FileName instead of a group?

Also, I did not figure out how to delete individual group entries from that DT_RetGroup List.

Regards,

Stephan
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    visualdatatools.com Forum Index -> Using External Programs All times are GMT - 3 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group