Optics and haptics: The picture 
 John M. Kennedy and Igor Juricevic
Psychology Department
University of Toronto 

Acknowledgements: Dejan Todorovic, Christoph Landerer, Josef Perner, Juan Bai, Shadi Edalati.

Abstract

Pictures are tactile as well as visual. Outline pictures stand for the same kinds of surface features in touch and vision. Vantage point geometry is used by blind and sighted perceivers in pictures. Limits of pictures may be comparable for the blind and sighted, and transcended in useful ways. 
Introduction 
In keeping with a conference on the multimodality of human communication, the purpose of this paper is to show that some aspects of pictures are tangible as well as visual. Many means of communication are often considered to be exclusively visual or auditory, but the exclusionary view requires radical revision, the evidence from pictures tells us. 

While they are often considered to be creatures of vision, pictures are amenable to touch in many ways. While that broad interpretation of the evidence is unmistakable, the full extent to which pictures belong in more than one sense is still unclear. What requires explanation is not just tactile pictures. We still need to give a proper account of visible pictures. Further, like touch, vision brings idiosyncratic features into service when triggered by pictures. In principle each and every means of communication is likely multimodal, but also in practice each can evoke unique features of a sense to some degree, much as a translation can keep the gist of a message but modify some connotations. 

Examples from Tracy

It may be best to begin with examples. First consider some pictures drawn at our behest by Tracy, a blind woman, at the Museum of Modern Art, New York, in January 2002. Tracy was blind early in life, having steadily lost vision in infancy to the point of total loss at age 24 months, too early to learn to draw using vision.

Tracy's drawings were made with a ballpoint pen on a plastic sheet resting on a rubberized board, a kit which produces raised lines when the pen writes on it and makes the sheet pucker. Conveniently, the long thin lines of puckers are raised on the side from which pressure is applied.

Tracy was asked to draw a cube balanced on a point with three faces towards her. The orientation of the cube was demonstrated to her with an 8.5cm wide wooden cube. She initially drew two quadrilaterals attached along one side, but then indicated this was not satisfactory and began again. She proceeded to depict the cube by first drawing a Y shape, standing for the front vertex of the cube. She completed her drawing by adding to the Y to show the three faces of the cube in front of her. She produced three quadrilaterals. They were formed by adding two lines, in V shapes, to each adjacent pair of lines in the Y. The added V shapes stood for the rear edges of the faces of the cube. The drawing reveals good use of a vantage point. It also suggests Tracy can project features of an object at various orientations and depths onto a flat surface. 

To test her use of a vantage point and projection further, Tracy was asked to draw three objects (a cube, a cone and a ball) set around a rectangle, and to make the drawing from 5 different vantage points. This is known as Piaget's three-mountains task (Heller and Kennedy, 1990). The vantage points were, first, her own location, second, a location on the opposite side of the rectangle, third, a location on what was the left side of the rectangle from her and fourth, one on the right side. A final one was directly above the rectangle. Tracy drew the rectangle and put the three objects on it, to left, right and middle correctly for all of the 5 vantage points. For example, from her vantage point, she drew the cube (depicted by a square) upper left, the ball (depicted by a circle) in the lower middle, and the cone (depicted by a dot for the pointed top, with a triangle filled in with arcs below the dot) to the right and half as far up the page as the cube. Also, the further away an object, the higher up in the picture plane she drew it. In her drawing to show the array from the vantage point of someone sitting opposite her, the arrangement was perfectly left-right and top-down reversed.

Interestingly for the vantage point "above" the rectangle (and no longer level with the tops of the objects) she said that since the vantage point is high, the array of objects should be drawn smaller, and she did so. The array "from above" was drawn as 4.2 cm on a side, smaller than 5.6cm for the corresponding side in the drawing made to depict the array for an observer on the right, which was the drawing made just previously. The distribution of the forms is precisely in keeping with the changing vantage point. Further, what is further in distance from the observer is shown by drawing the form higher in the picture plane. The mention of smaller size with increased distance "from above" suggests polar projection. However, the rectangle on which the objects sat was always depicted by a rectangle, suggesting parallel projection and similar shapes.

To probe her use of polar projection, Tracy was asked to draw two rows of three glasses evenly spaced and receding directly away from her across the table top. That is, if each row were to be extended in straight lines, she would have been sitting midway between. All the glasses were the same size. She drew the glasses as U shapes. She used two large U shapes (2.1cm tall and 3.5 cm apart) for the pair of glasses near the observer, two smaller U shapes ( 1.4 cm tall and 1.9 cm apart) spaced closer together for the pair of glasses halfway across the table and two still-smaller U shapes (.6cm tall and 1.1cm apart), spaced even closer together, for the glasses furthest away on the table. The further the glass, the higher the U on the picture plane.

In the drawing of the rows of glasses, Tracy uses convergence. The drawing uses decreasing sizes of objects and decreasing spaces to deal with increase in distance. It is of interesting that the space from the base of the glasses in the lowest row of glasses to the bases in the middle row is 3.5 cm, but the distance from the bases of the middle row to the bases of the upper row is merely 3.0cm. This diminution with distance is in keeping with polar perspective.

Tracy's drawings include many features richly deserving comment. Her lines show corners where two surfaces meet, and may show the occluding borders of flat and curved surfaces. Her raised lines are small ridges with two sides, on an otherwise flat surface. Each ridge depicts a single border formed by changes in depth and slant. She considers the vantage point of the observer, only depicting the surfaces of the object that face the observer. She uses line shapes similar to the shapes of the object's surface boundaries, such as squares for surfaces of cubes. Her shapes may involve projection from occluding boundaries, as in using quadrilaterals for the faces of a cube, and circles for spheres and triangles for cones, though alternatively, these circles and triangles might be cross-sections.
Tracy's devices

It is especially intriguing that Tracy uses features of projection to the observer. One kind of projection is a kind of parallel projection in which rectangular surfaces of the object are depicted by similar shapes in the outline drawing. A second is polar perspective, in which smaller size in both the left-right and vertical dimension means increasing distance. Tracy's drawings of the cube and the three-mountains may be in either kind of projection. The rows-of-glasses drawing uses aspects of polar projection. 

Both polar and parallel projection allow height in the picture plane to depict depth. The higher the object in direction from the observer the higher it is depicted in the picture plane. As a result, besides preserving the left and right order of the three "mountains", the picture preserves their vertical directions from the vantage point.

Tracy has been interested in drawing since she was young. She reports that she is very largely self-taught. She tackles new pictorial problems such as drawing two rows of glasses without seeking answers from others, and her solutions are ones she devises on her own initiative. Tracy is representative of congenitally and early blind people, interested in drawing, with largely self-taught abilities, who have participated in our drawing studies in several countries (Kennedy, 1993), though Tracy is at the upper end of the drawing-development spectrum. If we trace the development of drawing to the advanced level Tracy exemplifies, the evidence suggests that we will find that blind and sighted children are remarkably alike in three ways: use of outline, use of shapes similar to the shapes of faces of objects, and use of projection (parallel and polar). Communication with pictures is indeed multimodal. 

Theory
Tracy's abilities with outline and projection systems may surprise many. Her drawings call for new theories of touch, outline and projection. 

Pictures are about the sources of our perceptual inputs. They represent those sources by replicating aspects of those inputs: optics and what is tangible.

Just as vision is the psychological science of the optic input, haptics is the science of what is tangible. Vision is to light what haptics is to resistance. "Touch" is the informal term for a sensory system included in haptics. Since light and resistance are quite distinct, many commentators have considered that representations making sense in one domain would be foreign to the other (Arnheim, 1990; Gibson, 1962; Hopkins, 1998, 2000). However, Tracy's pictures, and pictures from other blind individuals (Kennedy and Bai, 2002; Millar, 1994; Eriksson, 1998; Lopes, 1996, 1997) undermine that view. But how can outline operate similarly in vision and touch? 

Light fills an angle at the eye. Within the angle, it varies in intensity and spectral composition, or wavelength. These properties vary with time. In contrast, the variables of tactile resistance in time are pressure, or variation in weight, and area of skin covered. The variables of optics and haptics are not fully commensurate, far from it. Granted, angle subtended at the eye converts to area, by means of the amount of retina covered by the optic input. But intensity of weight (pressure) is not directly related to intensity of light in physics, and spectral composition has no sibling in resistance. The conclusion that seems to follow is that many creatures of vision should have no equivalent in touch. 

The size of the object providing the visual input is not directly related to the size of the retinal area covered, since much depends on the distance of the object from the eye. In contrast, what presses on the skin is the actual object itself, not a projection from the object. One might think therefore that vision is distal and touch is proximal. Pictures are like transparent windows on a distal world. But feeling the surface of a picture in raised form is not like looking at a pictured world beyond the depicting surface. It cannot generate an impression of a panorama beyond, one might conclude. 

But of course the influences on vision and touch are not entirely incommensurate. The sweeping claim about differences between the distal and the proximal is much too restrictive. The key is variation in time. Using the proximal, touch operates across time to discover the distal relations between objects. Touch is not just a matter of sitting at a desk and feeling a surface with some raised markings on it. Touch is more than the domain of a few finger movements. We take time to walk around and touch things. Touch investigates space around us. Touch provides information as we get up from the desk, walk around the room, wander around the house, and step into the garden and along a path. Touch involves a point of contact changing across time. Also, at any given moment, touch has a point of outreach, a base from which exploratory movements are made. So touch uses the proximal at any instant but discovers the distal across time, and thereby the directions of things from many vantage points (Millar and Al-Attar, 2002). Presumably these directions include the directions of objects from each other. A tactile picture can perhaps be taken as an indicator of the directions of things from a vantage point. This may require some sophistication on the part of the observer, but nevertheless be quite feasible for most adults. If so, touch offers a base for Tracy to employ forms of projection.

Operating across time what touch finds is the surfaces of extended objects. Only a fraction of the surface is touched at any instant, but what haptics registers is the surfaces and their size and orientation. Some are large and some small. Some face vantage points from which the observer reaches out, and some face away, so we need to reach around occluders. The border between what faces a vantage point and what does not is often quite precise, as in the case of corners of a cube. It can be precise too for rounded objects if the observer considers what faces a particular small point in space. An undulating surface that we feel in an extended movement occupies a set of directions from our location at any given moment. This set of directions is part of the group of impressions of the corrugated distal surface stretching away from us. The part to do with direction from a point may come to the fore and be useful for some observers exploring tactile pictures on some occasions. 

Line
Borders of surfaces are shown by Tracy in her line drawings of cubes, balls and the like. In these uses of outline Tracy emulates what has been present in visual art since Cro-Magnon simple line sketches 50,000 years old (Kennedy, 1993)

Cave artists discovered several uses of line in pictures, the core being the ability to stand for edges of surfaces. The same uses are deployed by line in pictures for vision today. Further cave artists discovered continuous surface edges could be shown by discontinuous hatch markings in pictures, that is lines with gaps in them. In addition, lines in cave art pictures are often formed by grooves. These have two sides, much like Tracy's ridge line but in reverse. In cross-section, a groove is like the two hillsides of a rift valley with a valley floor in-between. The two sloping hillsides are plain to see. But though there two sides, the referent of the sunken groove in the outline drawing is a single border, and like Tracy's raised ridge line with its two sides, means a single change of depth or slant. 

Just as lines in pictures can be of several kinds -- black pencil lines, white chalk lines, ridges, grooves -- there are many visual inputs from the real world being depicted that create impressions of borders. Consider the many kinds and how they relate to the surface edges which lines can depict.

What creates visual borders has to do with luminance, colour, stereo and motion. An intensity contrast between solid black and white is a luminance change, and readily gives rise to perception of clear, sharp, continuous borders. Visual area V1 handles apparent orientation of such borders and could play a key role in detecting many of their properties. V1 also has subareas relevant to purely binocular borders. Instructively, V1 stereo cells help support perception of apparently-continuous borders linking visual elements that are physically spatially separate. That is the input has gaps but the stereo percept has an illusory contour that appears continuous. These seemingly-continuous borders readily look like the edges of surfaces. Likewise, area V2 helps produce a strikingly-continuous kind of visual border, called a purely subjective contour, from spatially-separate luminance borders. Given a monocular, static pattern, with suitable luminance variations, V2 performs this feat of extending continuous illusory contours across gaps, thereby linking physically-present contours. The illusory contours resemble infinitely thin wires, or they can appear like occluding edges of a single foreground surface, or corners formed by two abutting surfaces. Areas of visual processing beyond V2 likely contribute to the illusory contour looking like a surface edge. 

Small-scale motions of luminance borders can also be grouped in regions. A swarm of insects, or a school of fish, contain individual items making small scale motions. Given sufficient density, they suggest a body of a definite shape, but one without continuous borders. A key brain area for small-scale motion is V3. The point to stress is that these motions can give rise to an impression of linked or grouped visual elements that are actually spatially separate. Further, the motions of spatially separate elements can produce subjectively-continuous contours if elements accrete and delete at a border. The subjectively-continuous linkages may seem to be corners, or occluding boundaries of surfaces, depending on the motions around them, much as a herd of individual animals moving up and over the brow of a hill could define the hilltop as the occluding boundary of a rounded surface, or a crowd of skiers vanishing behind a mound can define the white snow-covered mound's perimeter against an equally-white snow-covered ski-slope. 

Another kind of border is made from a pure colour change. These strongly influence V4, and can be modestly effective at suggesting surface edges. Another area (V5) may be able to group elements and make visual borders that look like surface edges if the visual input is large-scale motions of individual luminance borders. To test for large-scale motion perception without small-scale motion, an element can vanish in one visual location and burst into view at another visual location, as if passing through a tunnel. The locations would only be only a degree or so removed in small-scale motion. Also, to obtain clear apparent borders from large-scale motions, the element may need relatively straight contours and sharp corners. 

In sum, each of the cortical areas V1 to V5 may contribute information about borders of surfaces, including the perception of change of surface slant at the border (a corner), or change of depth at the border (an occlusion edge). An implication of occlusion and corners being evident is that each border-making cortical region may contribute towards impressions of surfaces indicating the observer's vantage point, and as a result they may be governed by a geometry of depth to some extent. 

Borders and vantage points

It is an open question how V1 to V5 not only define visual borders but also use the geometry of vantage points and projection to handle depth, slant, occlusion and the observer's locus that is implied. Indeed, every single visual area relevant to visual borders may use a projective geometry such as perspective. After all, polar perspective is the exact geometry of the variation in depth of objects from a point. It is clear that a projective geometry is important in getting information in stereo vision. Physically, the same basic geometry applies to motion in depth, since motion of the observer through an object-filled environment yields a field of objects changing their directions around the observer.

Each visual kind of border may trigger a brain area with its own version of projective geometry. Or, alternatively, they may feed a set of borders like a flat spiderweb to central regions of the brain. Perhaps only in the innermost core of the central perceptual computer do the individual surface features begin to trigger perception of the perspectival relations between them. Only finally and centrally do they yield perception of a landscape. The issue remains to be settled. Indeed, how generally vision uses the properties of polar projection is still unsettled (Kennedy and Juricevic, 2002). The most likely account would be that each of V1 to V5 contains its own perspective geometry, but the geometries are similar in major principles, only disagreeing on matters of precision. One way to test this would be to show pictures of squares, with varying amounts of foreshortening, triggering V areas selectively. Each area might accept a certain range of foreshortening, but the range might differ from V1 to V5. 

Just as vision involves brain regions reacting to different kinds of borders, haptics may be served by distinct brain areas specialized for different aspects of tactile borders. 

Much like the visual cortex, the cortical regions for touch can be divided into a primary input-reception region, S1 (with several specialized layers), an adjoining region S2 to which S1 feeds signals, and further regions to which S2 sends signals. The regions are highly interconnected. Much like the visual system, touch is organized hierarchically, with more complex pattern processing following on the initial and simpler processing, and sending feedback to the simpler processing stages. 

Here, we will only present possible specialties to illustrate the argument. Borders on the skin between regions with different pressure may be served by some cortical area we shall call SX with cells that respond to the specific textures in the regions. The next stage, SY, might contain cells responding to the border itself, that is the changes at a border. Kennedy (2000) describes an assembly of cells that can detect such borders. SY might also be especially sensitive to different directions of motion on the skin. Vibrations (pressure variations in time) may have different effects in SX and SY, and borders between them. Attention to a skin region may affect SX's sensitivity much more than SY. Anticipation of a stimulus may actually decrease sensitivity in SX more than SY, and play a role in tuning the cells for effects of active exploratory motion (Chapman, 1994). Certainly, some tuning to information about locations in space being attained successively because of motions of joints, tendons and muscles is especially relevant to sensorimotor input. Theory of the relation between touch, attention and motor control holds that touch is part of a system that incorporates modulation in sensitivity due to predictable inputs from active motion, and modulation in input across time due to movement of the stimulus (Chapman, 1994). Further, touch incorporates cells with selective sensitivity to motions in particular directions, much like vision's regions. But how sophisticated is the analysis of space in S1, S2 and adjoining regions? It is hardly credible that the initial cortical analysis of tactile input would contain information about surfaces and perspective variations. But this kind of analysis must be performed somewhere that is open to tactile input. 

Tactile information is related to motion of the body, information about the direction of targets, and input from surfaces (D'Angiulli and Kennedy, 2001). This means the S1 and S2 input from a few pressures at the fingertip as we palpate (or our toes as we walk) at a given moment is related to an integrated pattern from the bodily platform carrying and controlling the fingers (and toes). It is this mobile platform and its postures that contains spatial information writ large. 

Vision begins with variations in light and produces percepts of surfaces. In physical dimensions, the input is not commensurate with the product. They are of different kinds, physically. Touch too begins with variations, though for touch these are pressure variations related to postural information. But it also yields impressions of surfaces. Touch, like vision, deals with edges, slant and depth, providing percepts of surfaces and their borders. As in vision, the currency of touch is the orientation of surfaces in space, including change in slant and depth, over which touch can range. Pressure in input, as it is related to mobile postures of the body, is largely exchanged for surfaces and their locations in the percept. The products of the haptic and visual systems have much in common. Vision's products imply the observer's vantage point, and so too do touch's. As in vision, a tactile concave corner involves an observer enclosed by surfaces, an occluding edge implies a front surface facing the observer and a rear surface is facing away. 

If touch's products in perception imply an observer, they also imply projection of some kind (Kennedy and Bai, 2002). But what kind? A front surface faces toward the vantage point, but does this involve polar or parallel projection? Either one fits the bill. Likewise, recognizing that the rear surface faces away from the observer could entail polar or parallel projection. Tracy may deploy different projection geometries, polar and parallel. It seems likely that she appreciates some aspects of a variety of projection systems, and the issue then for her is what is suitable on a given occasion. Of course, her savvy with projection systems is normally used in the real world. What is challenging in drawing is how to bring that knowledge to bear on projection onto a flat surface. Tracy, like most sighted people, is still working out the implications of the use of a projection surface. What is evident is that she has ready use of a vantage point, projection of similar shapes, and is able at times to apply polar projection to the overall size of objects and the spaces between them. She realizes that in principle further distance implies convergence. The change in distance means a difference in the relative directions of two objects, with angular subtense shrinking. Likewise the difference in direction of two parts of an object, such as the left and right sides, lessens with distance. 

Outline and links

In addition to speculation on the origin and geometry of borders, there is a key matter of representation to be addressed. Both visual and tactile borders can be depicted by lines. How is this done? What is most enlightening is just how different the line in the perceptual machinery can be from the physical input on a page that supports it. The input line can have two sides and yet what is depicted is one change of depth or slant. The input line can be made of dots with spaces between, but the referent's border can be depicted as continuous. Evidently there is a function in perception which treats some property present in many kinds of lines as also present in many kinds of surface borders.

A swarm of insects has a border, though the insects are separate elements. One way to define the border is to say the low-frequency variations (the sorts present in a defocused picture) indicate the rough layout of the swarm. High-frequency variations are highly localized ones, like the continuous, abrupt borders in hard-edged graphics. Should we say an outline provides a high-frequency border? Perhaps not. An outline picture can be made of fine dots, far removed from continuous hard-edge graphics. Alas, outline is also far removed from the low frequency variation that can describe the location of a swarm, since a single line of fine dots will not show up in a low-frequency (defocused) image.

One intriguing possibility is that tiny elements such as dots can define an area enclosed by the line, and the area itself can be taken as a low-frequency variation. Dots define lines, lines demark areas, and areas are interpreted separately from dots or lines. This successive-step model is a promising beginning. It goes through a stage in which spatial links between dots are processed, captures some high-frequency input (continuous abrupt borders), before accepting these as boundaries of regions, and then defines regions as broad or defocused swathes of the input. This model could be applied to touch as well as vision, since touch can detect dots and note the relation between them. While this is a promising start, it cannot be the key story however. The defocused end-stage is not the proper terminus. Outline readily shows precise spatial features such as nostrils and earlobes in a drawing of a head, or twigs in a drawing of a tree. These details are lost in defocused images. Therefore the kind of level of analysis needed is the one evident when we consider short links between tiny dots. A further caution to note is that the links themselves can be the representational element, as in a drawing of a twig. The regions bounded by the links are relevant to occlusion edges and corners, but we should not forget twigs, stick figures, wires and the like where the line itself depicts a linear foreground referent. 

A useful feature of the discussion of links is that links between dots are lines without thickness. The dots they join may have appreciable width, but the link is widthless. This may help us understand the function in perception which treats the width of a line, between its two sides, as immaterial. In outline drawing, two sides of a line show one surface edge. When we look at a cross-hatched line in an outline drawing, we accept what is discontinuous as equivalent to something continuous. The links between the hatchings are what does the work. 

Perception's procedure for finding widthless links may be an asymmetric function. We do not usually treat one side as equivalent to many, or what is continuous as equivalent to what is discontinuous. In standard practice, the individual contours and dots in the perceptual input trigger an impression of particular relations between the group of elements, a particular impression of linkages we might call the gestalt of a line. This is an asymmetric function because the elements trigger a grouping function, and this resulting group or gestalt does not trigger perception of an element. 

The gestalt of the set of linked elements can have the form of a single linear group. But grouping alone is not enough. The linear gestalt is the form which is in common to all outlines. It is this which acts as an outline and supports the perception of corners and occlusions. The gestalt function operates in both vision and touch, clearly, since drawings by the blind resemble drawings by the sighted. That is, it is the linear grouping produced in both touch and vision is a continuous link between separate elements even if the physical input is discontinuous, one dot visually spaced apart from the other, or each dot touched one at a time by the exploring hand.

One way to achieve apparent continuity in touch is for the tactile gestalt line to appear as continuous, much as a subjective contour from V2, stereo or motion can look continuous. Alternatively, the dots can simply be grouped as a line, much as three equally-spaced dots can appear as vertices of an isosceles triangle without the triangle sides appearing as faint lines. That is, the modus operandi of a gestalt line may be to allow detection of the gaps between the dots but have the gaps deemed to be irrelevant, as if the dots are taken to be information for a continuous referent. The dots represent continuity but do not constitute it. The dots in this case mark the location of a continuous contour, but the contour does not appear, only its location. In short, we need to develop theory, first, on grouping and machinery in perception that can handle relevance and irrelevance in different ways that might produce gestalt lines, and second on how the location of a gestalt line in turn might then be taken to be a representation of a continuous surface edge. At present all we wish to do is point out the issues that need to be solved.

Perspective

Besides outline, Tracy's pictures have in common with pictures for the sighted vantage point geometries. In particular, her pictures include use of convergence, that is diminished sizes of forms depicting objects, and foreshortened distances between the forms. These features make eminent sense to vision, but here they are in her tactile pictures. Evidently a general theory of projection onto a picture surface is needed, one more general than vision alone. In searching for such a theory, caution is advisable. While much may be in common to vision and touch so far as perspective is concerned, much may also be unique to vision or touch. Consider some distinctive factors, some to do with the angles subtended by the objects in a scene around us, and some to do with linear extents on a picture surface showing a scene.

As objects recede from us, the angle they subtend at the eye diminishes. Artists contend with this fact in drawing landscapes. It is telling that artists often have to resort to procedures such as holding up paintbrushes to estimate the relative visual angles of two objects with any accuracy. The reason is we are often mislead in judging the visual angles unaided unless the objects are close together in direction from our vantage point. The further apart the objects are in direction the more error creeps in. Vision often regards two equal visual angles (a dimension measured in degrees) as strikingly unequal, for reasons to do with the linear size of objects (a dimension measured in centimeters). Likewise, a more-distant object (like our left hand, held off to our left) that is making a small angle at our eye can seem to subtend the same angle as a nearby object (like our right hand, held off to our right) since it is the same size in cm, even though the near one subtends say twice the angle. 

In sum, when subjects look at objects their impressions of angle-subtended are contaminated by optical information for linear size (Juricevic and Kennedy, 2001a). There is no known direct equivalent for this in touch. When we point at objects from afar, the farther the object the smaller the subtense of the top and bottom of the object so far as touch is concerned (Kennedy, 1993). It seems unlikely that in comparing the angles subtended by two objects, touch would benefit from having the two objects in the same direction, and thereby escape a bias towards linear size ratios. Vision's accuracy depends on the objects being close in direction, even if the objects are far apart in distance from us. Touch is likely to have distance as a key influence on accuracy, but not proximity in direction. Touch would readily compare two objects 180 degrees apart, simultaneously exploring the two objects, one per hand. 

Binocular vision obtains excellent information about linear size and distance of objects. It responds to differences in the visual angles projected by an object to the two eyes. The differences in visual angles constitute information about linear sizes. Binocular vision obtains one percept, but bases it on two different visual angles. Since the two visual angles are not available in the final percept, it may be no surprise that judgments of the relative visual angles of objects are poorer in binocular vision than monocular vision. They are especially strongly biased towards linear size information (Kennedy and Juricevic, 1999). 

Vision makes use of many relations produced by polar projection. To detect the relative sizes of objects vision may make use of "meridian rules." The visual scene contains many similar objects and we can assess trios of these by seeing how the tops and bottoms line up. The imaginary line formed by aligning parts of two parallel objects in a scene we can call "a meridian." For example, three vertical columns on a flat surface may have tops in a line and bottoms in a line. If the two lines converge to the horizon of the flat surface, then the three objects must all be the same size. If all the bottoms line up but the top of one is head and shoulders above the line joining the other two, it is taller. This kind of "lining up" of bases of objects on a plain defines the meridians of the landscape. follows that if the line joining the bases of two columns, and the line joining their tops converge to the horizon the objects are the same size (Sedgwick, 2001). Informative meridians can be chosen freely from any objects and provide information about the relative size of the objects. It may be that the lining up of a trio of parallel objects (or a pair and convergence to the horizon), not only gives us the information that they are the same linear size, but can bias vision to some extent towards the misleading impression that their visual angles are similar.

Vision takes in a sweep of 360 degrees around us. But when we come to draw, or compare visual angles of objects in pictures, we select a fraction of the full panorama. In particular, we treat some objects as to the fore (directly in front of us) and others as to the side (left and right of the objects to the fore). In comparing visual angles of objects, our skill with pictures and projection may intrude. We may envisage the objects projecting onto a pictorial screen, as if we were preparing to sketch the object on a window through which we can see the object. The window to the fore is likely perpendicular to the line from the observer to the objects. A screen to the side may be an extension of the screen to the fore, or be turned so as to be perpendicular to the line of sight. In either case, it may be further from us than the forescreen, and compensate for the diminution of the object with distance. In judging the object's visual angle, we may act as if we are influenced by the size of the object's projection on the screen. We may act as if we estimate the fraction of the screen covered by the vertical extent of the object, for example. The upshot would be the visual angle of objects to the side would be overestimated by vision. Indeed, objects to the side (and more distant than objects to the fore) are seen as having relatively larger visual angles than is correct (Juricevic and Kennedy, 2001b). Asked to judge the relative visual angle of objects to the fore and to the side, we make judgments that approach the linear size ratio of the two objects. 

Apparent size of pictorial elements

Linear size's influence on apparent visual angle likely plays a role when we try to perceive the elements on a picture surface. Vision scans the lines on picture surface from a distance. Touch explores them with direct contact. The elements are distal for vision, and proximal for touch. Does this mean vision is subject to error and touch must be accurate? Far from it! Touch has geometrical illusions much like vision (Millar and Al-Attar, 2002). That is, opportunities for misreading the elements are present for both senses. Perspective patterns on the surface may influence attempts to detect the actual lines on the surface. The picture may be taken as a setting for a element and to the extent it depicts a scene the detection of the line element may be compromised. "Crosstalk" is a term Sedgwick (in press) uses for this source of illusion. 

A point to note carefully here is that crosstalk may differ from sense to sense. Both senses may use one set of geometrical principles, but once the main function is achieved there is room for different ancillary effects. Vision has striking impressions from perspective pictures, and touch may not achieve the same precision of depth perception from pictures. Rather, touch is likely able to emphasize direction, where vision may emphasize depth, for example. Vision is strongly affected by the optical information for the linear size of a depicted object. In crosstalk, this information affects the apparent visual angle of an object in a scene. The optical information for the apparent length of lines on a page is in many respects the same as the optical information for the visual angles of the object depicted by the line. Hence, the optical information for the linear size of a depicted object may influence the apparent length of the lines on a picture surface, and vice versa -- the optical information for the size of an element on a picture surface may affect the apparent size of the depicted object. Since subjects misjudge visual angles, influenced by linear size information, they should misjudge the size of the pictorial elements too. However, the frame of the picture and the pictorial surface texture may help limit errors in judging the element's linear size. If so, the crosstalk may particularly strongly affect the depicted object's apparent size. The element's size may affect the apparent size of the referent. 

The pictorial elements are just parts of pages that correspond to visual angles projected by object parts. For every misjudgment of visual angle there should be a comparable error of pictorial element judgment. Just as separation in direction allows the linear-size effect on visual angle to grow in vision, the illusion of differences in size on a picture surface is enhanced by separation (Juricevic and Kennedy, 2001b). Vision may be more prone to this error from depicted linear size than touch, and show the effect of separation in direction more strongly than touch.

Touch and imagined projection screens 

In one respect projection screens may be considered alike by vision and touch. When we draw objects, we often try to draw the form that would be present on a projection screen, and we envisage a screen at some location between us and the object, on which the object is projected. Often the screen we imagine is on the normal to the line joining our vantage point to the object. That is, objects to the side and objects to the fore are not imagined projecting onto one single continuous flat screen. But when we draw, we usually have no such choice. Typically, we are drawing on one continuous flat surface. The consequence of the mismatch between our imagined screen and our drawing surface's orientation would be too much use of foreshortening in the drawing (Kennedy and Juricevic, 2002). Consider: All surfaces parallel to a flat continuous surface project without foreshortening in the sense that perfect squares and circles project as squares and circles. Let us align the front faces of several cubes. Consider drawing the panorama of these objects on one flat picture. The front faces are parallel to the imagined screen for an object to the fore. They are drawn as squares, and not foreshortened. But front faces on cubes considerably off to our side are now tilted so far as our line of sight to them is concerned. If so, they would be drawn as foreshortened on the panorama by mistake.

We suggest this same error of overuse of foreshortening will be made by those using polar projection in touch as well as vision. The blind can envision forms of projection. But they may be as likely as the sighted to entertain a projection screen perpendicular to the line joining their vantage point to the object. A cube set much to their side, and a cube directly in front of their vantage point, will seem to present "front faces." Let the cube to the side have a face in the same plane as the cube to the fore. That face for the cube to the fore will seem to be a front face. But the aligned face will not seem to be a front face for a cube much to the side. The face deemed to be the front face will be closer to perpendicular to the line joining the cube to the observer's vantage point. 

In sum, the typical blind and sighted subject who turns to polar projective drawing has only a very partial understanding of projection. It seems likely that Tracy's pictures are similar to many a sighted adult's level of skill. Also, perceptual borders are triggered by different kinds of input. Do each of these follow polar and parallel geometry in their processing? The lesson from Tracy's drawings is that touch as much as vision -- very different kinds of input -- may be subject to roughly similar geometrical principles. It seems that apparent shape may be similar despite major differences in input. The importance of the vertical, left and right, facing towards and away from, the location of the observer, the role of depth and angle subtended -- all of these may play similar roles in coping with diverse inputs.

Limits and tropes
It may be helpful to mention briefly the limits of a system such as outline or projective geometry. Line may be used similarly in vision and touch, and many aspects of projection to a vantage point may arise in both senses. But the implication is that many of the limits of one sense may be akin to the limits of the other. If so, this provides the conditions for tropes in human communication to cut across sighted and blindness status. 

In language we might say "the man had a heart of stone" when we are searching for literal words to describe the man's lack of fellow-feeling. We overcome our verbal limitations by using a metaphor. Similarly when we find we cannot draw the wind, or smell or pain or motion we may be tempted to turn to a pictorial metaphor. We may draw the spokes of a wheel as transformed into concentric circles. We may draw paths of sounds bouncing around a room to convey echoes. In kind, we may draw the stony man as metallic.

These pictures we may consider to be going beyond a limit for literal depiction. Intriguingly, such pictures beyond the limits can be devised by the blind and the devices they invent are interpretable by the sighted. The implication is that devices that lie beyond the range of literal re-presentation can employ apt features. The features have some literal function, just as the word "stone" has its literal use, but some of the features can be put to use in a setting in which the referent of a literal term does not belong. In the new setting, the device refers to some of the relevant features of the object, and a pictorial metaphor is born, interpretable by the blind or the sighted (Kennedy and Merkas, 2000).

Summary
Pictures are a form of representation that can be haptic as well as optic. Line in visual and tactile pictures can depict depth and slant features of surfaces. Some aspects of the observer's vantage point and projective geometries are available to the blind and the sighted, and can be expressed in pictures. The limits of pictures provide opportunities for tropes intelligible by both the blind and sighted. Communication by pictures can use more than one modality. The implications are diverse and call for new theories not only of pictures but also of vision and touch. 

References