Canvas

 

 

A Canvas is something which can be drawn on and supports basic drawing operations.

Every Canvas has a model, which is asked to draw itself on a canvas, and react to events.

 

The Canvas API needs enhancing to support:

• Sub-canvases.
• Canvas "needsRedraw" status.
• Event handling.
• Double-buffering.
• Canvases presenting transformations (clipping, rotation, zooming, warping) of other Canvases.

 

I would change Canvas by:

 

• Allowing a canvas to have movable sub-canvases. These would map 1:1 to windows in the X window system, or cached bitmaps in VNC, or display lists / textures in OpenGL. These could be moved around the screen
• Canvases could be implemented as bitmaps or vectored graphics; the application doesn't need to know what implementation is actually used.
• Introduce a "needsRedraw" system of some sort. A Canvas implementation may or may not cache its contents (as a bitmap or vectored graphics/display list). Various implementations may discard the cached contents at times, or perhaps not even cache content.
• Use micrometers rather than pixels as the unit of measurement and provide a "pixelPitch" method to return the size of a pixel. For example, my screen has a pixel pitch of 282 micrometers. A 600dpi printer would have a pixel pitch of around 42 micrometers. You could use a SmallInteger to store micrometer values.
• Introduce, somehow, an event system closely coupled to a Canvas (because some events have coordinates relative to a canvas).
• Allow canvases to use other canvases for: double-buffering, clipping (i.e. show part of another canvas in a bounding box on this canvas) and scrolling, transformations such as zooming and rotating (?), showing images (i.e. have a primitive ImageCanvas which loads from bitmapped data), one-off plotting (have a bunch of flyweight canvases to render fonts). The Canvas could replace the Form.
• Canvases are transparent by default.

 

Here, the "drawer" is the object which draws (as opposed to the furniture item).

 

Child canvases

 

• Thought: Child canvases might actually need "Views" as glue. ParentCanvas->View->ChildCanvas. Views could then decide what of the following child behaviours are used...
• Another thought: Views might be Canvases.

 

One of the enhancements would be to add child canvases. Child canvases could be used in many ways, for example:

 

• They could be "windows" (frameless) that would appear "on top of" a parent canvas but be clipped by the parent's extent. The contents of the canvas could be cached by the so that the child can be moved across the screen without needing to redraw itself.
• They could be used for scrolling inside a parent that manages some scroll bars. The child canvas would be clipped and only have a portion of itself visible inside the clipped extent on the parent. By changing the part of the child canvas that is shown and having it re-drawn, the child canvas would appear to scroll inside the clipped boundary the parent provides.

 

Sub-canvases would be useful for caching bitmaps and moving them across the screen smoothly. For example, a 2-D scrolling game could be made using sub-canvases to draw the actors and layers of the background.

Sub-canvases have a z-index.

 

 

Sub-canvases could support transparency to be very cool.

 

A sub-canvas is clipped to the bounds of it's parent canvas, for security. This prevents a canvas drawing over controls that do not belong to it. A sub-canvas acting as a window would have a relative position to it's parent which it cannot change (but the parent can).

 

Sub-canvases are hidable or could be rendered in multiple places (e.g. for flyweights).

 

One sub-canvas would have the keyboard focus and would receive keyboard events.

 

A canvas might be special, e.g. an OpenGL canvas for 3-D graphics, or an MPEG canvas for hardware accelerated movies, or perhaps an image canvas that only takes a bitmap and can be used as a child canvas.

 

Every sub-canvas has a small mouse-pointer canvas which is what is used to draw the mouse pointer when it is hovering over that canvas.

 

A FormCanvas has other FormCanvas's as sub-canvases.

 

Every canvas can have a location attached to it that will be visited if the user clicks that canvas. A clickable link would then be a transparent sub-canvas with a location attached to it.

 

Types of Sub-canvases:

• The usual, drawable with a position. These would need to be of the same type as the parent. A FormCanvas has child FormCanvases. A PostscriptCanvas has child PostscriptCanvases.
• FormCanvases could be rendered either as a stored bitmap, a shared parent bitmap with a set of occluded rectangles or a display list.
• Special canvases, such as GLCanvases, may not be supported on all canvases (even though ideally they would be. OpenGL on Postscript anybody?).

 

How would you create a GLCanvas on a FormCanvas?

 

FormCanvas addChildCanvas: (GLCanvas new: 10000@10000) ??

or

newCanvas := FormCanvas addGLCanvasAt: 10000@10000 size: 20000@20000.

 

Perhaps a Canvas can have a fall-back rendering mechanism? If a particular parent canvas does not support the child canvas's rendering methods, it can fall back on either raster or bitmapped graphics mapped to the parent?

 

Perhaps use a factory somehow - make the parent canvas be a factory, or something?

 

 

Drawing

 

Possible types of Canvas include:

• XCanvas rendering to an X Window System server (at last!).
• Win32 canvas rendering using whatever there is on Windows. Ditto for Mac.
• FormCanvas, of course.
• Rome, using the Cairo library.
• PostscriptCanvas, rendering to a .ps file. Ditto for printing APIs?
• SVG canvas (cf: PostscriptCanvas).
• VNCCanvas, rendering to a remote VNC client.
• GLCanvas, using OpenGL.

 

Several of these canvases support native sub-canvases (XCanvas with Windows, GLCanvas with display lists, VNCCanvas, ...). Those that don't can emulate this.

 

Several support antialiasing (Rome, GLCanvas, ...?).

 

Some support transparency (GLCanvas). XCanvas notably doesn't, although there may be modern extensions that allow this, or the library could hack around this.

 

Some have native font support (XCanvas, Win32, Rome).

 

GLCanvas can also be 3-D, meaning that a sub-canvas stack can really be in 3-D. This API is limited to 2-D except for the stacking nature of canvases; a 3-D API would be more general (and real 3-D buttons that stick out would be very cool).

 

A Canvas could also do fancy effects: sub-canvases may drop shadows on the canvases below them, or cast a subtle light. It would be very cool if the shadow respected the transparent parts of a canvas. The canvas with active keyboard focus may be more illuminated than others. Canvases might be slightly reflective or have reflective parts, making them reflect the canvases above them. Fancy effects like this are much more suited to a proper 3-D environment than a 2-D drawing API.

 

For now: The Canvas implementation does it's best to render commands given to it. If it does not support anti-aliasing, then anti-aliasing is simply not done. If it does not support transparency, that is not done either. Programmers using canvases just need to be aware of this and not rely on transparency or antialiasing.

 

Because various renderable objects have a lot of properties, it might be better to render a particular object on a canvas rather than add special methods such as #lineFrom:to:colour:. These objects can be made immutable so that the canvas can cache them (if the canvas is of the caching sort).

 

Lines have:

• Start, end points, or for a path, a series of points.
• A stroke width.
• A colour (including alpha).
• Edges either being square or rounded. If a line is part of a path, then the corners may be rounded or flat.
• Lines could be part of a longer path. The path might be enclosed or not.
• dots / dashes?

Maybe lines are really rectangles - they have a width, a length and a fill colour!

 

 

Dots must have a circumference. They are really a circle.

 

Bezier or other curves?

Circles, eclipses, arcs?

 

 

Paths (rectangles, polygons) have:

• A series of points; between these points are lines with the properties above.
• Enclosed or not.
• A means of rounding the corners (round / flat / none).
• A means of rounding line ends (flat / round).
• If enclosed, a fill colour or gradient.
  • Is a gradient a special type of colour? A gradient would need coordinates, so it would be specific to a canvas.
  • Is a colour specific to a canvas (e.g. colour mapping, colour correction, number of colours).
• Perhaps the fill could be another canvas? Perhaps we could have a gradient canvas and colour canvases?

 

If fills and solid colours are actually special canvases, then perhaps they could be replaced with another canvas, such as an image?

 

A lazily drawn Canvas could then be used for any fill. This would also be useful for viewports. A canvas can be lazily drawn using a dirty-marking mechanism that asks the renderer of the canvas to redraw it.

 

 

Coordinate system

 

What coordinate system should a canvas use? Currently they use pixel-based coordinate systems, with the top-left corner being 0@0 (or 1@1?).

 

The options are:

 

• Pixel-based. This gives code completely control over how pixels are placed at the expence of resolution independence - e.g. this has no meaning in a plotted environment and isn't convenient on a 600dpi printer canvas. The common problem here is when users upgrade from their 640x480 VGA screens to high-res 1920x1280 17" LCDs - they can no longer read their text because the pixels are so small.
• Point-based, using "pt" or post-script or typography points. These are about 0.3mm, which is a silly unit and just a bit bigger than a pixel.
• Metric units with floating-point coordinates, e.g. using millimeters.
• Metric units with Integers, e.g. using micrometers. This has the advantage of using plain Integers for operations but still with high fidelity.

 

If a pixel-based system is used, then coordinates could either represent the center of each pixel, or integral coordinates representing the edges between pixels.

 

My preference is either of the metric units. These are device and resolution independent. Operations on pixels are usually basic operations - adding, dividing, comparing, so the same code would work with either floating point or integer operations except for equality comparisons.

 

An integer-based system could have a method that returns the size of the pixel (e.g. 257 micrometers), so pixel-by-pixel control can still be achieved with multiplication. For example, on my current monitor, each pixel is 282 micromillimeters (exactly!). If the GUI API provided a method for returning the pixel size (both horizonally and vertically) then an application can choose to align graphics with pixels, or alternatively just ignore them and use absolute coordinates instead (e.g. 100mm x 200mm).

 

Juan Vuletich's Morphic 3 has a very flexible coordinate system.

 

--> Leave this for later - for now, just use whatever Canvas already provides. Implement an improved UI device in version 2.0.

--> Or maybe this should go in; version 1.0 should provide a uniform graphics API.

 

Coordinate origin location

 

Should 0@0 be at the top left or bottom left?

Arguing for bottom-left:

• Cartesian coordinate systems use the bottom left.
• Measuring an object sitting on the "ground" (e.g. a font glyph) means measuring it from the bottom up. In meatspace, gravity is a downwards force.

 

Arguing top-right:

• User interfaces and documents flow from top to bottom, so the top-left is the starting point. However, this only applies to European scripts. Other languages have right-to-left text.
• The "gravity" of a page is an upwards force; objects will tend to congregate at the top of the page with empty space at the bottom of a page.
• Resizing a window usually means that the Canvas in that window would be clipped from the bottom and the right sides, but the canvas would be moved if the top or left sides were moved. This puts the origin in the top left.
• Java 2D, Cairo and the existing Squeak Canvas use the top-left corner as the origin.

 

This is really an arbitrary decision. Whichever approach is taken will have only a minor effect on code.

 

Models

 

Canvases typically have a model. The model is the target for events, and is asked to redraw the canvas.

 

Subcanvases each have their own model. The redrawing of a sub-canvas does not influence the redrawing of it's parents, and the drawing of a parent canvas does not influence the drawing of child canvases. Each is updated separately, although perhaps by the same thread for efficiency.

 

When an application starts up, the main canvas is passed to the model, which then adds sub-canvases to the model. For example:

 

SiteBrowser>>navigateTo: aSite

currentCanvas reset. "remove all sub-canvases, remove all steppers, clear contents"

aSite setUpCanvas: currentCanvas.

currentCanvas markDirty: (currentCanvas bounds).

MySite>>setUpCanvas: canvas

canvas setModel: self. "Start receiving events."

canvas addSubCanvas: (set up a sub-canvas here).

MySite>>drawOn: aCanvas bounds: bounds

"The bounds is the area that needs redrawing; I can choose to redraw just that, or everything. "

... canvas drawing commands...

 

 

Redrawing the Canvas

 

A Canvas can be persistent; it survives screen refreshes. An application can safely hold a reference to a Canvas and draw on any part of it at any time.

Every Canvas has a model which it can invoke "drawOn: canvas bounds: bounds" to have that canvas redrawn. The bounds exist to redraw a part of the canvas, such as for example when that canvas is only exposed through a clipping canvas or is occluded.

 

Some ways of caching canvases would be:

• For BitBlit on the display, each FormCanvas contains the bits it consists of already.
• For OpenGL, a display list or a texture on a rectangle would be redrawn by the graphics card and wouldn't need callbacks to the drawer.
• For VNC, the client (on the user's side) could cache bitmaps.
• For The X Window System, I believe it's possible to store drawing info on the X terminal?

 

If the Canvas is cached in a display list or bitmap, then redrawing is not necessary for most move/expose operations. If the Canvas does not store state, then a redraw may occur whenever the canvas is exposed on the screen (e.g. brought to the front) or moved.

 

The Canvas needs to know when drawing is complete. One way is to mark the area on the canvas as "clean". Another is to return successfully from the drawOn:bounds: method.

 

Event handling

 

 

Mouse events have a screen location, which is a point position on the canvas that they occurred on.

Redrawing a sub-canvas could occur as an event. Redrawing would only occur when a canvas is damaged by some other canvas moving over it.

Keyboard events don't have a particular position. Keyboard events would be sent to the canvas which has "keyboard focus".

 

Events such as the morphic "stepping" architecture could be done externally to this.

 

Types of events:

• onKeyPress (key, shift/ctrl/alt/modifier status)
• onMouseMove (mouse buttons up/down?)
• onMousePress (split up into several methods for each button? The application can do that.)
• onDoubleClick? I don't really like double-clicks.
• onNeedsRedraw (list of damaged areas)
• onMouseEnter, onMouseLeave (mouse button status)
• onKeyboardFocusEnter, onKeyboardFocusLeave.
• onDrag, onDrop ? (pass something in which a small dragging canvas can be attached to which sticks to the mouse pointer?)
• onCommand (cut/copy/paste/print/poweroff/pause/play etc - keyboard commands defined by the UI).
• onCanvasMoved (?), onResized.
• onCanvasDestroyed, onCanvasCreate ??
• onShow, onHide ?

 

Events regarding redrawing or resizing need to be separate. Resizing events will cause the GUI framework to resize each widget.

 

 

Commands

 

Some modern keyboards have special keys for many functions, e.g. "zoom in/out", "print", "back" etc. These could be mapped to "commands" which are sent to the canvas's model, as an event, to be interpreted as that particular command.

 

Canvas classes

 

The root class of the Canvas hierarchy has:

• A size (in micrometers)
• A pixel-pitch (in micrometers).
• A model, for reacting to events and redrawing, or null if events should be ignored.
• keyboard focus (possibly managed outside the canvas).

 

Some Canvases have:

• Cached display state - a bitmap, display list etc, which could be local or remote.
• Child canvases, each with a position (rectangle, z-index) relative to this parent.
• A RGB order for LCD screens.
• Antialiasing info.
• Font management of some sort???

 

Some rough ideas:

 

• BasicCanvas - does not support child canvases.
• ParentCanvas - supports child canvases... or views...?
• ChildCanvas - is a child canvas. Maybe not needed if Views are used?
• ClippedView - a view of a canvas which can be placed on a parent canvas. This shows a portion of another canvas.
• WindowView - a view of a canvas which can be moved around like a window. Has a z-index. Could be implemented using ClippedView (superclass maybe?).
• StaticView - a view which gives a snapshot of another canvas.
• UpdatingView - a view which constantly updates? This would use the event system.
• BitmapCanvas - a canvas that takes a given bitmap in a constructor like FormCanvas. Doesn't necessarily have vectored commands.
• VectorCanvas - a canvas that can be drawn on using vectored graphics. Bitmaps can be added using Views (maybe??).
• InteractiveCanvas - does events...?
• TransformingCanvas - warps the drawing commands of the child canvas, e.g. FishEyeCanvas, RotatingCanvas etc, ala Morphic3.

 

Perhaps:

 

Canvas - bounds, position

• subclass ParentCanvas. This would contain child canvases.
  • subclass Canvas (FormCanvas or RasterCanvas). This can load images too.
  • subclass VectorCanvas / GLCanvas. This would add images by converting FormCanvases to textures?
  • ...more canvas implementations
• subclass ClippingCanvas
• subclass SubCanvas ??BitmapCanvas|

 

 

e.g. the ClippedView:

 

ParentCanvas -> ClippedCanvas -> ImageCanvas.

 

The ClippedViewCanvas has coordinates for rendering on the parent, and coordinates for which part of its child canvas is rendered. An application would do drawOn: the ImageCanvas, and the results would be visible on the ParentCanvas. How this is implemented depends on the platform. E.g. for a BitBlt implementation, maybe there are renderOn: methods which are called from the parent canvas down to redraw the UI?

 

The user of the Canvas does not need to worry about the actual rendering to the screen. The Canvas architecture does that - the user of the canvas is concerned with drawing on the canvas when it receives a "please redraw on me" event.

 

If you have a TransformingCanvas that sees through to a canvas with children, those children should also be transformed.

 

 

Speciality Canvases

 

• 3-D Canvases, such as OpenGL.
• Canvases on 3-D elements inside the 3-D canvas.
• MPEG Canvas for playing movies.
• High-speed image manipulation canvas: rotating, zooming, etc.
• Low-speed image manipulation canvas aka Gimp canvas supporting e.g. gaussian blurs, high-quality zooming, other techy image manipulation.
• Hardware device-based canvas, e.g. showing a TV tuner.
• External canvas; i.e. another OS window.
• Externally implementated Canvas, e.g. FlashCanvas, HTMLCanvas, GTKWindowCanvas which can have GTK widgets added to it (although it wouldn't be drawn on?).

 

Some of these won't accept drawing commands - does that mean they aren't Canvases? Perhaps Canvas needs a superclass called maybe Window which has sub-windows (and Canvas does not), or perhaps Window is a separate class of which the above are superclasses?

 

Text

The Subcanvas API will probably also require a text API, because text can be handled differently by different APIs.

 

See Pango tutorial: http://www.ibm.com/developerworks/library/l-u-pango1/ for some text rendering concerns.

 

Text is rendered using:

• Squeak's own text rendering using FormCanvas.
• Pango, Cairo on a Cairo-based canvas.
• Vectors in OpenGL.
• X fonts in X.
• Postscript fonts (Adobe Type 2? OpenType?) on Postscript, or maybe convert them to vector graphics?

 

OpenType seems to be a well understood font format.

 

Kerning is best done on a per-paragraph basis (see TeX), so it might be necessary to have a rectangle/paragraph of text. The width would be set; the height would depend on the text. Kerning would be done by the canvas rather than by the application. Paragraphs need to support centering, full-alignment, right/left alignment, text direction etc. Paragraphs would need to support mark-up: colours, bold/italic/small-caps/underlining,

 

The application should also be able to specify non-wrapped text for a single line; perhaps by setting the width to zero?

 

Font metrics need to be exposed to the client for special effects such as highlighting up to a particular character, or for putting fancy underlining on characters.

 

All sizes should be converted to micrometers.

 

Text could be either rendered using pre-build featherweight glyphs, or by rendering each character individually to the subpixels.

 

The API needs to have enough primitives to allow the advanced user to implement his own rendering. This could be as simple as rendering individual glyphs. One possibility is to make text layout a custom package.

 

FreeType only renders individual glyphs and provides metrics about them. FreeType generates bitmaps, so the Canvas only really needs to support some BitBlting mechanism with font rendering and text layout being done externally.

 

...perhaps I could only include support for rendering a particular glyph at a time? But then ligatures would not be supported...?

 

If Postscript stores text as a string and does it's own kerning, then it is more efficient to make an API which can handle whole paragraphs.

 

Testing

 

It would be nice to have a test screen of some sort, like this: http://en.wikipedia.org/wiki/Image:Pm5544-768.png.

 

This could be showen when the system is busy booting or loading up.

 

This needs testing:

• Edges of monitor (or paper) correctly aligned (think badly configured CRTs).
• Fonts are sub-pixel antialiased properly. Note that LCDs can have different sub-pixel arrangements, and could be rotated 90 degrees for portrait mode.
• Pixel pitch correctly configured (show a ruler showing centimeters).
• Pixel pitch correct in both dimensions (show a series of circles of various sizes. Think badly configured widescreen TV).
• Lines (vertical / horizontal) are straight (think CRTs with a fishbowl effect going on).
• Show the current time, animated.
• Show sub-canvases of various sorts, animated. Or perhaps this is unnecessary - this should be a separate test.
• Show colour capabilities - show full pallette if possible. E.g. if the display only has 256 colours.
• Show actual pixel sizes along screen so it can be seen how big the screen actually is.
• Perhaps show off screen performance? Or maybe we don't want to consume too much CPU.
• Show that colours are accurate: black, white, primaries (RGB, CMYK).
• Show monitor sensitivity: Show very very light gray lines getting darker, ditto for black lines. Maybe combine with a colour pallette?
• Show various other capabilities of Canvas?
• Show an actual picture (think: demos of printers in shops).
• Perhaps show system information: version of VM, hostname, memory, etc?
• If used for booting, show a progress bar / log?
• Show events - mouse movement, mouse clicks, modifier keys, keypresses, (commands?).
• Perhaps it could include / be replaced by a console?
• Show a logo?
• If good unicode support is supported in the release, show that off?
• Show motion blur / response time.
• Look on the Internet for good Monitor calibration images.
• Don't include tests that might induce an epileptic fit.
• Include multiple tests for different monitor sizes, from 160x160 4-color screen to 29275x21025 600dpi A0 paper. When screen size is reduced, reduce the number of tests included.
• Test for flicker when double-buffering is not happening (e.g. alternating lines of pixels, somehow?).

 

http://www.lagom.nl/lcd-test/

 

Links

 

Lessphic: http://piumarta.com/software/cola/canvas.pdf

http://en.wikipedia.org/wiki/Windows_Presentation_Foundation

See the Java APIs for Swing and Java2D for inspiration.

http://www.antigrain.com/research/font_rasterization/

http://cairographics.org/

Font rendering and text handling: http://www.pango.org/

http://www.w3.org/TR/css3-webfonts/

 Create your own free wiki!HelpPublic, not yet premium Privacy Policy RSS