/* $Id$ */ [ ] CVS #* [ ] [cv.split] [ ] reestablish 'gzfile' feature in a better way. (gzfile is disabled for now) [ ] bugs roman [ ] add claude's fixes [ ] ... [ ] [fps]: translate into pd: "publish if fps>0.001" (to remove outliers) soon: * removed reference.*ml * videodev.c should be upgraded to V4L2 asap. +-------+-------+-------+-------+-------+-------+-------+------- Common Video Framework : [ ] a common library for all video IO (cameras, files, web), which is the project of Thomas H and Johannes. [ ] a six-way converter for GEM/PDP/GridFlow. There are already converters between GEM and PDP and between GEM and GridFlow but there is none between PDP and GridFlow and then some things could be easier to install and run in a cross-platform way. [ ] a way to make GEM/PDP/GF conversions *implicitly* instead of having objects for that. [ ] threads for video? [ ] GEM geos vs 3DP: how do they compare? [ ] is there anything else deserving to be unified between the video plugins? [ ] videojack [ ] http://gstreamer.freedesktop.org/ +-------+-------+-------+-------+-------+-------+-------+------- GridFlow 0.9.0 : [ ] 266678 bytes of Ruby left to delete [ ] switch from ruby to tcl [ ] distinguish between C++ types Grid* and GridFlow* [ ] replace source_filter.rb by SWIG completely [ ] make [#draw_points] [#draw_rect] [#edit_polygon] [ ] redo [#rotate] as abstr [ ] Salsaman recommends supporting Videojack because of unicap support which means getting DV support for free. [ ] sync detection with -ltcl* detection (still buggy) [ ] tcl.pd_linux that does all of the same as gridflow.pd_linux but for tcl instead of ruby. [ ] rewrite MétaRuby Type.rb in tcl ? [ ] expand poe.tcl system as necessary: C-based objects, AOP, static methods, SELF inheritance, ... [ ] look at camorama +-------+-------+-------+-------+-------+-------+-------+------- See Also : [ ] Generic Image Library: http://opensource.adobe.com/gil/html/giltutorial.html [ ] OpenCV +-------+-------+-------+-------+-------+-------+-------+------- [ ] [for] has a bug in "from". [ ] #out x11 doesn't accept 0-sized grids [ ] #print doesn't accept 0-sized grids [ ] [display] bug, can't print nested lists, due to Array===[] returning false (???) [ ] #inner doesnt support seeds of size>1, and isn't vecop-compatible, and ... [ ] #inner could optimise permutation matrices? ClaudiusMaximus matju: [#draw_image] inlet 0 method "op " is undocumented, but very useful :) ClaudiusMaximus matju: [unix_time] is maldocumented, outlet 0 is a grid Dim[28](uint8), not a symbol [ ] pd_examples -> examples [ ] update doc [ ] replace the unimplemented "r" numtype (Ruby) by an implemented "t" numtype (Tcl) [ ] ditch xml (switch to helppatches) [ ] claude wants http://en.wikipedia.org/wiki/Graham_scan malloc alignment = 4 mod 8 (en utilisant valgrind) ClaudiusMaximus matju: feature request: [#store4], behaves like store but accepts grids of floats as coordinates and interpolate using 4-point interpolation, eg, right inlet -> (3 3 f # 0 0 0 1 1 1 2 4 8), left inlet -> (1 # 0.5) outputs (0.5 0.5 0.5) (or whatever the interpolation would really be), left inlet (2 # 1.5 1.5) outputs (3) back to the [display] object, it lingers for a moment after deletion and returns a 'pd_activate' error after I move it matju: once I delete it, unlike the other gui objects, it takes about one second before it dissapears, maybe a difference in tcl versions? http://oprofile.sourceforge.net/about/ also: qprofile, amd profiler, opreport -cl --demangle=smart [ ] cvs2svn [ ] -ftree-vectorizer-verbose [ ] try removing or modifying -falign on PPC/OSX [ ] encoding of quicktime RGBA only works with [#cast b] (is that still true?) [ ] reading tux.tga is completely broken [ ] install of libquicktime is very wrong on my system (is it 0.9.8?) [ ] make example on separable convkernels [ ] add concepts of hot inlet and conf inlet at the code level [ ] [display] should use clocks to update tk less often [ ] [display] leaks in server ? [ ] deprecation warnings [ ] Add a video latency patch (in examples) [ ] can't have FPatcher and GridObject behaviour at once ?? [ ] Numop: warn on misalignment [ ] remove GridInlet* ? [ ] configure chokes if GCC3 doesn't have the C++ plugin [ ] [#import~] [ ] [#export~] [ ] [#expr] [ ] [#line]? [#pipe]? [#delay]? [ ] #: option swap [ ] consider including [mysql] [renamefile] [ls] [shell] [ ] re-enable int64,float64 [ ] added #matrix_solve, #slice [ ] Copy-on-Write (COW) [ ] Fully hyperlinked manual [ ] internals documentation [ ] internal grid protocol redesigned [ ] raise limit of in/outlets from 10 to 100 or unlimited. [ ] add support for "anything" methods (def _0_(sel,*a)) [ ] look into -ftree-vectorize [ ] test.rb should become much more serious. (the @pack bug really sucked) [ ] look at Tetra Piccolo capture cards, as well as ImpactVCB (Hauppauge) [145] "list -1"->[@import] only tells error message once every other time [144] small leak remaining in x11. may have to do with non-use of XDestroyImage. [140] is [@convolve] actually cross-convolution? that is, convolves with a mirror image of the convolution matrix. [139] [#peephole] matju: if you try to move the GOPed abstraction the peepholes don't move along he :) better yet, the abstraction's bounding box's lower right corner will not move up or to the left of the low right corner of the peephole it contains. Instead it will resize itself as you move it around. [137] [gf] peeperr: problem with set_geometry, wrong size. [134] #import with zero size crashes? [133] numop ** can freeze GF [117] osx: [#store] crash instead of raising [107] MMX computations may be done misaligned, which may be bad. [105] when file not found, mpeg error messages are wrong, they say the filename is "". [102] osx: error: shmget failed (cannot allocate memory) in X11, for regular pictures [078] "unsupported codec" bogus errors. [097] bin/backtrace does not detect scripts [005] can't open "": Success; loading a nonquicktime file using quicktime [006] #in#open problem with String argument. [025] format_grid: write test for other bpv's [026] latency is not that good. (GC ?) [030] FormatGrid: not properly tested (crashes sometimes) [031] stdout should be redirected if bridge is present [032] error handling sometimes leads to crash [033] error handling shows inanely useless messages (sometimes) [034] format/mpeg3.c,quicktime.c might cause lockup on exit [038] some seeds aren't typechecked [039] jpeg file cat crashes on second file (jpeglib buffering?) [040] GRID_FINISH triggered while backstore ??? [079] can trap segfaults in codecs of libquicktime. [090] http://poynton.com/notes/colour_and_gamma/ColorFAQ.html#RTFToC30 [041] what would be a way to make GF work well with Pd's idea of a list? [045] write unit tests about required functionality [046] write unit tests about error checking [047] make system to ensure everything is working before a release. [048] potential issues between fork() and X11 ? [054] it's possible to crash by sending data in two inlets of a same object at once --- make the development process more test-driven: * file every feature request and bug report into base/test.rb so that gridflow fails those tests THEN make gridflow pass those tests. =================== Installation [ ] less compile-time checks and more run-time checks [ ] should GridFlow be bundled with some of its own dependencies? [ ] find a good profiler (not gridflow's internal...) [ ] cross-compile: http://www.libsdl.org/extras/win32/cross/ http://rooster.stanford.edu/~ben/linux/cross.php Documentation [ ] make a short feature list as a promo [ ] add credits [ ] indiquer comment faire un rapport à propos d'une carte v4l ou d'un mode x11 non-supporté [ ] document how to report bugs, basic troubleshooting [ ] document our distribution of the archives [ ] ascii conventions: [foo] vs "foo", etc. [ ] document internals [ ] describe indexmap in doc [ ] document backward compatibility issues [ ] architecture: class->type/shape->use->role [ ] heavily hyperlink documentation [ ] automate more parts of the documentation [ ] benchmarks (especially: simple video i/o) [ ] mimic [pdp_help_input] [pdp_help_output]: they are convenient --- remember to keep doc/help/screenshots/examples up to date Architecture [0.8] reentrancy of objects (to allow true recursion) [0.8] inlet-attrs (attrs associated to inlets) [ ] eliminate the Dim class, replacing by Grid itself (?). [ ] pull mode (functional-style) [ ] bidirectional messages (constraint/relational-style) [ ] send-reply messages (imperative-style) [ ] add reflection (#global:classes, #global:methods, ...) [ ] transmitting grids by diffs? [ ] dispatch on _#{i}_-named methods (any selector one inlet) [0.9] generalize the backstore [ ] Outlet#buffer should harmonise with the lcm of its corresponding Inlet#factor's. [ ] lazy grids: Grid#operator T* makes sure all the grid is received and Grid#range(int start, int n, T bogus) makes sure the specified part of the grid is received. [0.9] accounting: grids messages packets numbers bytes copybytes [0.9] memory profiling [ ] add a "query" class for debugging other objects? (would be able to extract extra info from each object/class) [ ] add a GridFlow::LaterException or something (which tells to retry later with the same packet because another grid reception is not completed yet.) [ ] interpret nested lists as n-dimensional. [0.8] MMX memcpy() ? [0.8] MMX _runtime_ autodetection Computation Objects [ ] #counter (similar to #for) [ ] #for "step" argument should have default {1...} [ ] zero option in #convolve (?) [ ] #conv (simpler #convolve) should allow n-D convolution [ ] #convolve rewritten as combination of #conv+#border ? [ ] #store: option zero (?) [ ] #import~, #export~ [ ] #fold, #scan: option reverse (right-to-left) [ ] #join: option swap [ ] #matrix_inverse, #matrix_solve [ ] #timeline (or rtmetro): sequencing of events from grid [ ] #replace_if, #random_select, #markov_chain, #sort, #slice, #splice [ ] #polygon_perimeter, #polygon_area, #draw_polygon smoothly [ ] fuzzy #store (interpolate) [ ] write a complete benchmark script [ ] spatial biquads (ask Tom) [ ] string operations to compensate for the feebleness of PD in that area. this includes [#sprintf]. --- see also APL/J/NArray/PDL/Aplus/Yorick/IPOW reference Effects [ ] #mosaic [ ] #gaussian_blur, #sobel, #pixelize, #despeckle (median) [ ] #dither, #draw_lines, #draw_points, #fire, #queue [ ] #chroma_key: i0=RGBpicture i1=RGBcolor i2=tolerance o0=RGBApicture with fuzzy flag; (don't call it #chroma_key though) maybe #mask... [ ] turtle (logo style) [ ] effectv [ ] create abstractions for usage patterns (eg: like #remap_image) [ ] see gimp for ideas [ ] kaleidoscopes [ ] bilinear interpolation a+(b-a)*x+(c-a)*y+(a-b-c+d)*x*y (#scale_by smoothly?) [ ] #scale_by/#downscale_by fractional values [ ] bicubic interpolation [ ] multilinear interpolation (n-dimensional?) [ ] #daltonism [ ] #perlin ? [ ] #moment could support more than 2 dimensions # told me to: # RGBtoYUV : @fobjects = ["#inner (3 3 # 66 -38 112 128 -74 -94 25 112 -18)","@ >> 8","@ + {16 128 128}"] # YUVtoRGB : @fobjects = ["@ - (16 128 128)",#inner (3 3 # 298 298 298 0 -100 516 409 -208 0)","@ >> 8"] Input/Output [ ] replace "open " by "open " [ ] videodev: is it possible to reach 60 fps ? [ ] a #store that is persistent (saved in .pd) ? [ ] Grid: support storage in all six types [ ] Grid: bpv 1,2,4 uint [ ] Grid: text version of the format [ ] Gimp, GStreamer, ffmpeg, xine, mplayer, PDP, OSC [ ] support for output inside of a GUI app (Tk/Gtk/Qt/...) [ ] get subtitling of tv channels [ ] add messages "load", "save" as open,bang,close combos. (?) [ ] SDL: should support multiple windows [ ] x11,sdl,aalib: use buffering to synchronise frames at a constant rate. [ ] x11: sync with the actual vsync of the monitor [ ] x11: maybe vram-to-vram copy using X11's Pixmap remote-objects [ ] shell object? [ ] HPGL, [unixtime] [ ] [parallel_port], [serial_port], [usb_mouse] Format [ ] remaining number type support in formats [ ] all: possibly get rid of the "format" notion ? [ ] PNM: P1,P2,P3,P4,P5 support [ ] XBM, XPM, GIF support [ ] fork: pseudoformat for launching format handler as separate process [ ] put_at/reassign in x11/sdl/quartz [ ] /dev/dsp, /dev/mixer [ ] Generic SHM/MMAP Drivers [ ] "open grid pipe" and such. [ ] multi: multi-file (by glob pattern, e.g.: foo*.ppm) [ ] AVI [ ] format grid: "headerless until 10" (text files) User Interface [ ] color picker [ ] #edit, grid edition dialog, etc. [ ] peephole: [ ] works without x11 (for osx...) [ ] does it really work with something other than 3 channels ? [ ] add a popup menu that makes sense [ ] vector slider (n parallel sliders in one, Dim(n)) [ ] 2d slider [ ] radial slider (angle chooser) [ ] drag-and-drop palette of object construction [ ] object construction menu specialized for each class (generated from class reflection/metadata) Extra/Examples/Etc [ ] .pd: read; write better [ ] examples/generique.pd [ ] add examples from the workshop, etc how about creating aliases like "open window" = "open x11 here" (or arbitrary user-defined sequence of messages settable in ~/.gridflow_startup) and "open videodev" -> "open videodev /dev/video0, option channel 1, option size 240 320" maybe there should be a separate ~/.gridflow_config that would be saveable from gridflow, and ~/.gridflow_startup would be edited only by the user but ~/.gridflow_config would still be a text file. i say that because the startup is a script so gridflow couldn't just do search-and-replace in there, because the option could be inside a "if" expression and such... at most, gridflow could _append_ text to .gridflow_startup, to be non-intrusive... Feature Requests From: Dave Bergevin [ ] synchronized audio with the video when using mpeg1 & 2 [ ] full screen mode (without the menu bars or frames) From: Alexandre Castonguay [ ] server side pd to handle multiple incoming commands, audio + video streams [ ] support for audio in the mpeg streams alx: le tab completion ne donnerait pas seulement les noms possibles pour le prochain argument dans le objectbox; il donnerait aussi une description de l'argument (surtout dans le cas où l'argument a beaucoup de valeurs possibles). je pense que le help de Pd est une bonne idée (le concept hands-on) mais ça pourrait aller plus loin. j'ai pensé intégrer le manuel de référence de gridflow directement dans le programme, disponible au runtime, dans l'éventualité d'un GUI qui saurait en tirer avantage. matju: détecter les formes; détecter les couleurs; support mpeg4; hotspots > http://redshift.sourceforge.net/debugging-GC/ [ruby-talk:53835] > You might also want to search the web for "malloc debugger". Tcl_GetObjResult Tcl_GetErrorInfo ... Tcl_GetStringResult may also be helpful. (note I can't seem to find Tcl_GetErrorInfo (maybe I dreamed it up)) matju: BTW CatchObjCmd might help too I see that the new Tcl_Dict commands have taken over the HEAD's CatchObjCmd ClaudiusMaximus reports that allocating a 197M grid causes a crash (turns out C++ throws exception, can't allocate, signal ABRT) ClaudiusMaximus says: I want to be able to send output from Pd+Gridflow to a video4linux loopback device so that other v4l programs can access the stream. I found the video loopback device here: http://motion.sourceforge.net/vloopback/ iSight on linux : http://episteme.arstechnica.com/6/ubb.x?a=tpc&s=50009562&f=8300945231&m=40110681 needs something called 'coriander'. the person who got it working mentions libdc1394. http://www.tele.ucl.ac.be/PEOPLE/DOUXCHAMPS/ieee1394/cameras/index.html: I learn that the iSight is compliant with version 1.30 of IIDC. the docs for coriander mention that the author needed to create a major and minor device for the camera. retyping (damn pdf's) -> there are two devices to create: raw1394 and video1394 raw1394 has major 171 and minor 0 while video1394 had major 171 and minor 16 any useful info? having a camera like the iSight supported could be fantastic for video capture/detection on a laptop. Not to mention that the bttv cards are getting harder to locate -mfpmath=sse2 remember to check for volatile VALUEs if GC problems ever occur Tom Schouten wrote: " added demo/rawvideo.pf and some support scripts in bin/pf-rawvideo.* this is an example on how to use external programs for raw video input. rationale: mplayer is a very nice piece of software supporting virtually any input format/codec. since i'm not planning to waste time on integrating more input codecs in pf, the best alternative is to have mplayer (actually mencoder) or ffmpeg as a subprocess. the same will be done for recording later. " ==6998== Mismatched free() / delete / delete [] ==6998== at 0x1B9016E8: operator delete(void*) (vg_replace_malloc.c:246) ==6998== by 0x1C0216DD: Pointer::~Pointer() (puredata.c.fcs:89) ==6998== by 0x1CDD7500: CObject_free(void*) (main.c.fcs:87) ==6998== by 0x1CC8DBBE: garbage_collect (gc.c:1183) ==6998== by 0x1CC8DE14: rb_newobj (gc.c:415) ==6998== by 0x1CCE74D5: str_alloc (string.c:61) ==6998== by 0x1CCE76EF: str_new3 (string.c:127) ==6998== by 0x1CC77539: rb_eval (eval.c:3687) ==6998== by 0x1CC7798A: rb_eval (eval.c:3305) ==6998== by 0x1CC76F69: rb_eval (eval.c:3320) ==6998== by 0x1CC7CC6F: rb_yield_0 (eval.c:4869) ==6998== by 0x1CC7DEE5: rb_yield (eval.c:4950) ==6998== Address 0x1BC3E9A0 is 0 bytes inside a block of size 20 alloc'd ==6998== at 0x1B901DF0: memalign (vg_replace_malloc.c:332) ==6998== by 0x1CDD5D37: gfmalloc (main.c.fcs:499) ==6998== by 0x1CDB7505: GridOutlet::begin(int, P, NumberTypeE) (grid.h.fcs:504) ==6998== by 0x1CDCD4EA: GridOutlet::GridOutlet(GridObject*, int, P, NumberTypeE) (grid.h.fcs:1018) ==6998== by 0x1CF5B4BB: void GridStore::grin_0(GridInlet*, int, Pt) (flow_objects.c.fcs:275) ==6998== by 0x1CF5EAE5: void GridStore::grinw_0(GridInlet*, int, Pt) (flow_objects.c.fcs:256) ==6998== by 0x1CDB59DA: GridInlet_begin_1(GridInlet*) (grid.h.fcs:921) ==6998== by 0x1CC6D589: rb_ensure (eval.c:5303) ==6998== by 0x1CDB8341: GridInlet::begin(int, unsigned long*) (grid.c.fcs:199) ==6998== by 0x1CF0DE7D: GridStore::_0_grid(int, unsigned long*) (flow_objects.c.fcs:404) ==6998== by 0x1CF15353: GridStore::_0_grid_wrap(int, unsigned long*, unsigned long) (flow_objects.c.fcs:404) ==6998== by 0x1CC6D88B: call_cfunc (eval.c:5480) rational numbers: http://wiki.tcl.tk/13877 very old bookmarks: http://pdl.perl.org/ http://www.opendragon.com/Pages/MaxObjects.shtml (not responding) gstreamer Arts Builder http://homepages.inf.ed.ac.uk/rbf/HIPR2/wksheets.htm http://freej.dyne.org/ FObject.subclass("rubyarray",2,2) { def initialize() @a=[]; @i=0; end def _0_get(s=nil) case s when :size; send_out 1,:size,@a.size when nil; _0_get :size end end def _0_clear; @a.clear end def _0_float i; @i=i; send_out 0, *@a[@i] if @a[@i]!=nil; end def _1_list(*l) @a[@i]=l; end def _0_save(filename,format=nil) f=File.open(filename.to_s,"w") if format then @a.each {|x| f.puts(format.to_s % x) } else @a.each {|x| f.puts(x.join(",")) } end f.close end def _0_load(filename) f=File.open(filename.to_s,"r") @a.clear f.each {|x| @a.push x.split(",").map {|y| Float(y) rescue y.intern }} f.close end } FObject.subclass("regsub",3,1) { def initialize(from,to) _1_symbol(from); _2_symbol(to) end def _0_symbol(s) send_out 0, :symbol, s.to_s.gsub(@from, @to).intern end def _1_symbol(from) @from = Regexp.new(from.to_s.gsub(/`/,"\\")) end def _2_symbol(to) @to = to.to_s.gsub(/`/,"\\") end #doc:_0_symbol,"a string to transform" #doc:_1_symbol,"a regexp pattern to be found inside of the string" #doc:_2_symbol,"a replacement for the found pattern" #doc_out:_0_symbol,"the transformed string" } FObject.subclass("memstat",1,1) { def _0_bang f = File.open("/proc/#{$$}/stat") send_out 0, Float(f.gets.split(" ")[22]) / 1024.0 f.close end #doc:_0_bang,"lookup process stats for the currently running pd+ruby and figure out how much RAM it uses." #doc_out:_0_float,"virtual size of RAM in kilobytes (includes swapped out and shared memory)" } FObject.subclass("rubysprintf",2,1) { def initialize(*format) _1_list(format) end def _0_list(*a) send_out 0, :symbol, (sprintf @format, *a).intern end alias _0_float _0_list alias _0_symbol _0_list def _1_list(*format) @format = format.join(" ") end alias _1_symbol _1_list } class FObject def self.gfattrs; @gfattrs={} if not defined? @gfattrs; @gfattrs end def self.gfattr(s,*v) s=s.intern if String===s gfattrs[s]=v attr_accessor s module_eval "def _0_#{s}(o) self.#{s}=o end" end def self.help gfattrs.each{|x,v| s = "attr=%-8s" % x s << " type=%-8s" % v[0] if v[0] s << " default=%-8s" % v[1] if v[1] # what default="false" ? GridFlow.post "%s", s } GridFlow.post "total %d attributes", gfattrs.length end def _0_help; self.class.help end def self.gfattrs; @gfattrs={} if not defined? @gfattrs; @gfattrs end def self.gfattr(s,*v) s=s.intern if String===s gfattrs[s]=v attr_accessor s module_eval "def _0_#{s}(o) self.#{s}=o end" end def _0_get(s=nil) s=s.intern if String===s if s then if respond_to? s then send_out noutlets-1,s,__send__(s) else ___get s end else self.class.gfattrs.each_key{|k| _0_get k } end end end there are bugs with abstractions that use [route]->[list] instead of [route2] because they fail when $1="list".