add `set terminal svg` and capture output to svg buffer

[mfrasca]

I guess this is the first place where intervention is needed:

I'm holding a spreadsheet in an emacs buffer, not yet decided whether to use ses or org., in both cases I have to go through gnuplot and in particular org-plot makes use of gnuplot-mode.

I'd like to grab the graph in a buffer, in svg format.
the svg format allows me further editing of the graph still using emacs.

my workflows using ses:

• one ses cell is reserved to a lisp expression evaluating to a string looking like a pair of vectors that I paste in the calculator buffer
• have the calculator generate the svg code in the *gnuplot trail* buffer
• collect the svg code in a svg buffer

the workflow using org-plot is more complicated:

• run org-plot/gnuplot
• this will generate a file in /tmp and open a window I do not need, so I close it.
• switch to the *gnuplot* buffer and copy the commands
• use gnuplot-make-buffer to open a new gnuplot buffer
• in the new buffer execute set terminal svg
• then run the commands copied from *gnuplot* buffer
• collect the svg code in a svg buffer.

I do not see svg in the possible terminals offered by gnuplot-mode.

when terminal is set to svg, I would expect gnuplot-mode to do this:

• filter the svg output coming from gnuplot and put it in a svg buffer.

first time gnuplot outputs svg code, it will be almost complete (misses the closing svg tag).
subsequent times it starts immediately after the closing defs tag.

[joddie]

It might be possible to implement this as a more general feature: Emacs could capture Gnuplot's output in any image format by using temporary files, and place it in a buffer. This would effectively allow you to plot into an Emacs window, and you could use image-mode to toggle between SVG source and rendered image.

In fact, this is already mostly implemented by the gnuplot-mode code that inserts inline images into the Gnuplot comint buffer. It would mostly be a matter of inserting images into a different buffer, and enabling different image types besides "png".

The lack of a closing </svg> tag looks to me like a buffering problem, FWIW. If you do set output "blah", plot something, and then set output, the file is flushed and the closing tag appears.

[mfrasca]

thanks for the reply! this quite simplifies the workflow using gnuplot-mode.

set terminal svg
set title 'salinity'
set ylabel "mS/cm"
set xlabel "m³/d"
set output "/tmp/b.svg"
plot '/tmp/org-plot18589dFU' using 6:3 with points title ''
set output

but I've never really programmed in elisp, so I'm a bit lost in your code, looking for what to change/add in order to automatize the process.

I guess I would like to do this:
when one sets the terminal mode to an image type, the output is diverted to its corresponding image buffer (unless explicitly diverted to a file). what happens now is that it goes to the gnuplot "comint" buffer.

so I would execute set terminal png and gnuplot-mode sees this and sets the output to the buffer *gnuplot-png*.
same for set terminal svg sending the svg code to the *gnuplot-svg* buffer.

is this viable? what keywords to look for in the code?

[joddie]

I think this is viable. I have a branch with a proof-of-concept implementation which I will push shortly for you to try out, if you like. I am still working on getting the buffering issue sorted out.

I had thought of having a single *gnuplot output* buffer, but your suggestion is possible too. Is it useful to have separate buffers for separate image types?

[joddie]

Please try out the prototype implementation of this in plot-to-window branch. In a fresh Emacs, load "gnuplot.el" from that branch, then do the following:

1. M-x gnuplot-show-gnuplot-buffer
2. From the "Gnuplot" menu, choose "Display plot output > In dedicated buffer" and "Display plot output > SVG images"
3. Plot something as normal
4. You should get a new pop-up buffer called *gnuplot output* containing the generated SVG file. In my Emacs (without SVG support) it is in nxml-mode. If you have SVG support, it might show up as an image, in which case you could use C-c C-c to toggle between viewing the source and rendered.

[joddie]

Note: I've never used org-plot so I don't know how this feature should interact with it. If we can first get this working when plotting from the Gnuplot interaction buffer and gnuplot-mode buffers, that can be tackled separately.

[mfrasca]

with the current changes I can indeed get the graph in a svg buffer.

how can I visualize/log what org-plot does with gnuplot-mode?
with the present changes, org-plot does not manage to generate output consistently.

if I run org-plot/gnuplot, the possible outcomes are:

• it just works (in particular if I run it as first thing).
• it uses gnuplot-send-hiding-output to report "Selecting deleted buffer" (nothing else happens)
• it hides the output buffer, says 'Buffer "gnuplot" has a running process; kill it?' (and then it works, showing again the output buffer or it hangs, with the spinning mouse until C-g)

[joddie]

Hmm, that is strange. It may help to do M-x toggle-debug-on-error, which should give you a backtrace on errors which you could copy and paste here. However, this does not tend to work very well for filter functions, which run asynchronously.

Could you post a minimal org-mode example as a Gist, plus a recipe to use for testing?

[mfrasca]

me behind very intermittent internet connection. was not logged in when I created this:
https://gist.github.com/anonymous/697852a0ab35a15f208c
updated
https://gist.github.com/mfrasca/330b268fa62b390ec2dc

[mfrasca]

is there a different way to reset gnuplot?

(when (get-buffer "*gnuplot*") ;; reset *gnuplot* if it already running
  (with-current-buffer "*gnuplot*"
(goto-char (point-max))
(gnuplot-delchar-or-maybe-eof nil)))

this asks for user intervention.

[mfrasca]

I'm looking at the code of org-plot/gnuplot... one point I notice is that it tries to do some cleanup relying on an idle timer. the cleanup doesn't happen. do you think it could be possible for gnuplot-mode to offer callbacks, that could be called after specific events in gnuplot-mode?

[mfrasca]

is it OK to invoke gnuplot-send-buffer-to-gnuplot at the end of a with-temp-buffer block?

[joddie]

1. Thanks for the Gist recipe. I have pretty limited time this week, but I will try to look into this.

2. There is something funny going on with handling the gnuplot buffer and process.

3. Yes, it should be OK to have gnuplot-send-buffer-to-gnuplot within within-temp-buffer.

4. Adding more hooks is a good possibility, but there is another problem. I don't see how the line

(run-with-idle-timer 0.1 nil (lambda () (delete-file data-file)))

in org-plot can work without lexical-binding. The binding for data-file is long gone by the time the idle timer runs. It should probably be:

(run-with-idle-timer 0.1 nil #'delete-file data-file)

[joddie]

Once i made that change to org-plot, I seem to be able to plot your table repeatedly using C-c " g without problems. Does that also work for you?

[joddie]

Hmm, no, it's still possible to get the "Selecting deleted buffer" error. Investigating..

[mfrasca]

I suspect the buffer being deleted is *gnuplot* (if I have a window visit it, invoke org-plot/gnuplot, the window will switch to a different buffer)