visualdatatools.com Forum Index visualdatatools.com
Discussion for DataTank and DataGraph
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

scatter graph point color from dynamic range of third dim

 
Post new topic   Reply to topic    visualdatatools.com Forum Index -> Feature Request
View previous topic :: View next topic  
Author Message
ijstokes



Joined: 27 Jan 2010
Posts: 24

PostPosted: Fri Jan 29, 2010 12:28 pm    Post subject: scatter graph point color from dynamic range of third dim Reply with quote

It is rather laborious to setup scatter graph point coloring. It would be nice if I could specify a column to use as the coloring base and then a color map, and have DG automatically do something like:

1. given scatter graph vector V = (X,Y) describing (x,y) pairs, and vector C selected for the coloring, then the scaled vector S is on the interval (0,1), where S = ( C - min(C) ) / ( max(C) - min(C) ).

2. given a color map vector M, v_i from V selects m_j from M where j = floor(s_i * size(M))

Color maps could then be of arbitrary size. The common Matlab colormaps would be a really nice start.

Adding in a color legend would then be necessary as well, where the displayed range of colors is mapped against C, not against S.

If you really wanted to get fancy with this, the color distribution on S could be weighted by "density": assign a region of the colormap range to values of C not based on the value, but based on the relative position and number of entries around that value. E.g. if C were gaussian distributed, then ~60% of the colormap (from the center) would be allocated in the range of +/- 1 std_dev(C). In this scenario, D = sort(C), with {d_k in D}, and

S = {s_k | s_k = k/size(D)} -- i.e. we don't care about the value of d_k, only its relative location in the sorted list D.
_________________
Ian Stokes-Rees
Harvard Medical School
Back to top
View user's profile Send private message
David
Site Admin


Joined: 25 Nov 2006
Posts: 1904
Location: Chapel Hill, NC

PostPosted: Fri Jan 29, 2010 10:02 pm    Post subject: Reply with quote

The colors right now are just for discrete colors (color scheme) but I am planning to make Colormaps for continuous values. DataTank does that for example (only that).

What I am planning on doing is to make the range part of the color ramp definition, just like the color scheme is now. Would you want a continuous color ramp, for example varying shades from green to red, gray scale etc.

One more thing that I'm planning for 3.0. I typically go up to the nearest .1 when I add a command and up to the next 1.0 when something bigger changes. This would require a beef-up of the parameters since now the parameter definitions might get a lot more involved and preferably interconnected.

David
Back to top
View user's profile Send private message Send e-mail
ijstokes



Joined: 27 Jan 2010
Posts: 24

PostPosted: Fri Feb 26, 2010 11:31 pm    Post subject: Reply with quote

I'd be looking for a color ramp, to provide a visual queue for a 3rd dimension, that would auto-magically scale based on the max/min of the 3rd dimension to select the color from the map.

E.g. in this image:

http://abitibi.sbgrid.org/~ijstokes/drop/DG-RightClickProblem-selects-table.png

The last graph on the right in purple shows protein sequence similarity (x-axis) versus a structural match score (y-axis). There is an essential 3rd dimension which is not visible: the size of the match. There are two proteins here, and the sequence ID is only for the "aligned" region. A large region with a high % of matching sequence is very different from a small region with a high % of matching sequence, but right now we have to laboriously construct discrete colors based on ranges of alignment size, or use the filter mechanism to only show structures with a large aligned region.
_________________
Ian Stokes-Rees
Harvard Medical School
Back to top
View user's profile Send private message
David
Site Admin


Joined: 25 Nov 2006
Posts: 1904
Location: Chapel Hill, NC

PostPosted: Sat Feb 27, 2010 1:37 am    Post subject: Reply with quote

So the issue is that it is tedious to create a Color Scheme where you specify 0-0.10 is dark blue, 0.10-0.20 is lighter, 0.20-0.30 is lighter still etc?

with the match coming from the basealigned column?

Currently Color Scheme is geared towards specifying schemes based on matches that can be of various forms, strings etc. This is flexible but tedious for certain very common cases.

I am thinking about a continuous color map but my nagging worry is that a continuously varying sliding scale is not what makes sense in most cases because you really want to categorize things. I'm not against doing that, but I think that what is needed more is a coloring scheme that makes the following simple:

- Select a thematic style (rainbow, blackbody radiation, two color etc)
- Select number of distinct colors
- Select a range.

Here the range would have a suggest button that would go through the commands that use the color scheme and pick up the range that would look good at that time. So it wouldn't be completely automatic but I've found that changing things based on the range would make it hard to use the same color map for two commands and also lead to mistakes where a certain hue will represent different value based on the data.


This would not be that hard to add since it would still be discrete values and the color legend would have the same form as you have now. The main change would be that instead of the color scheme menu having the entry "Create New Scheme" at the end, it would have two entries one for the old style and another like "Create Banded Scheme" or something like that.

Is this along the lines of what you want? Could you send me a data file where you have gone through the slightly mind numbing exercise of setting match to be (a,b], range to be values like "0.1,0.2" etc and pick a coloring scheme that you feel is helpful.


Later I will see if it makes sense to add a continuous variation of this.

David
Back to top
View user's profile Send private message Send e-mail
ijstokes



Joined: 27 Jan 2010
Posts: 24

PostPosted: Mon Mar 01, 2010 12:32 pm    Post subject: Reply with quote

It is more than tedious, it is really impractical. I am used to relying on Matlab colormaps of 64, 128, or 256 colors. Creating this kind of range by hand in DG would take forever. Plus it is very common in Matlab (and many visualization systems) to play around with the color map to come up with one that has a dynamic color range over a region of interest. In Matlab you can programmatically build the color map. In DG a similar system could be put in place by having a mapping function that would do an f(z) translation on the colorization dimension, and then use the proposed DG auto-colormap for color selection from f(z) (where z is the 3rd dimension selected for coloring).

When the 3rd dimension is a real value, and possibly on a dynamic range (i.e. not predetermined) then it is important that there is a system that can do color selection from an (effectively) continuous range of a colormap.

I'll send a link to a file with a manually constructed color range when I have time. Sorry I can't do this now.
_________________
Ian Stokes-Rees
Harvard Medical School
Back to top
View user's profile Send private message
David
Site Admin


Joined: 25 Nov 2006
Posts: 1904
Location: Chapel Hill, NC

PostPosted: Mon Mar 01, 2010 11:39 pm    Post subject: Reply with quote

DataTank does only smooth color maps, and even though that is good for scalar fields it is not as good for categories. I might eventually add those as color maps to DataGraph, but I think that in most cases you should not sample from a smooth gradient when coloring points. The main reason is that the eye really can't distinguish that many colors when they are not up against each other or next to other colors. Here is an example of what can happen:

Now try to figure out what the color numbers are. This is made by a DataGraph file - http://www.visualdatatools.com/DataGraph/phpImages/TrickyColor.zip The trick is that the colors on the right are all the same. The color ramp on the right is copied from DataTank and there the color is computed analytically (and for the rainbow color map there are going to be more than 256 colors).

The point is that you could set up a gradient and use it to color points but you are really not going to be able get that much information from it. Of course you are unlikely to get a gradient background, but this also applies when you have varying density of points, other colors etc. The eye is a very good difference engine but isn't all that good at picking out intensities.

That said, in the latest beta I am working on making it easier to set up gradient like color maps. The color map entry now has a gear menu, like I use in a number of other places, and this will use information from the column or columns that you actually use with with the color map inside the drawing commands. I look for a few patterns in the data, and I'm very much looking at making it smarter. If you send me a sample data set I can make sure that it does a decent job in very few clicks.

David
Back to top
View user's profile Send private message Send e-mail
stschiff



Joined: 04 Mar 2009
Posts: 34
Location: Cologne, Germany

PostPosted: Thu Feb 16, 2017 7:14 pm    Post subject: Reply with quote

I just saw this post and wanted to know whether there is any new development on using color ramps as a third dimension in Scatter Plots?

Now that the beta has useful color ramps for scalar fields, and even customisable color ramps, it would be awesome to simply be able to select a color ramp as "Color Scheme" in a Scatter plot. Here is a plot I made with Mathematica for that purpose, but I'd much prefer Datagraph for this, as Mathematica drives me nuts when it comes to polishing and combining plots:

https://www.dropbox.com/s/ltkhesufg8ulc7v/scatter_plot.pdf?dl=0

You are certainly right that this is often not the recommended way to plot things, I agree that one should categorise when possible, but in this case it makes sense (you just have to believe me Smile )

Thanks,

Stephan
Back to top
View user's profile Send private message
David
Site Admin


Joined: 25 Nov 2006
Posts: 1904
Location: Chapel Hill, NC

PostPosted: Mon Feb 27, 2017 11:34 pm    Post subject: Reply with quote

I added this to the latest beta.

David
Back to top
View user's profile Send private message Send e-mail
stschiff



Joined: 04 Mar 2009
Posts: 34
Location: Cologne, Germany

PostPosted: Tue Feb 28, 2017 4:22 pm    Post subject: Reply with quote

Thanks heaps! I think there is still a bug, though. Some points don't get plotted now... I'll send you an email with a script file and two plots.

Stephan
Back to top
View user's profile Send private message
David
Site Admin


Joined: 25 Nov 2006
Posts: 1904
Location: Chapel Hill, NC

PostPosted: Wed Mar 01, 2017 1:11 am    Post subject: Reply with quote

The latest beta fixes this issue. And thanks for pointing this out Embarassed.

David
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Post new topic   Reply to topic    visualdatatools.com Forum Index -> Feature Request All times are GMT - 3 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group