Rendering line-drawings with limited resources

Rendering Line-Drawings with Limited ResourcesStefan Schlechtweg,Thomas StrothotteDepartment for Simulation and Graphics,Otto-von-Guericke University of Magdeburg Universit¨a tsplatz2,39106Magdeburg,Germanye-mail:{stefans|tstr}@isg.cs.uni-magdeburg.deAbstractPictures play an important role in the communication municative goals can be expressed for in-stance with the specific use of different levels of abstrac-tion in pictures.Especially in textbooks and scientific publications,images are used where the level of detail of different objects is deliberately changed to express the importance of an object in the picture.In this paper we introduce the concept of“drawing re-sources”for rendered line-drawings.This concept de-fines measures for controlling the level of detail of sin-gle objects in a drawing.We present a system which is based on a line-renderer and which enables a user to interact with the drawing thus created.The concept of drawing resources yields a new basis for formulating the user’s ideas of the appearance of thefinal image. Keywords:non-photorealistic rendering,line-drawings,level of detail,abstraction.1IntroductionThe emphasis of rendering techniques has traditionally been on producing a detailed,complete image of a ge-ometric model.This seems to be a narrow view on computer graphics because more or less abstract im-ages often better serve the purpose at hand.“If the ultimate goal of a picture is to convey information”, as Foley et al.pointed out[1],“then a picture that is free of the complications of shadows and reflections may well be more appropriate than a tour de force of photographic realism”.Line-drawings are particularly suited to support abstraction.We propose to control algorithmically the level of detail,and with it the level of abstraction contained in a computer generated line-drawing.More specifically,we introduce and study the following questions:•Are there any possibilities to measure the degree of detail in a line-drawing?•How can you—if you can measure it—change the level of detail in accordance with the measure-ments?Besides rendering a more or less abstract line-drawing, we propose new interaction facilities for editing images. We argue that a close relation between the underlying geometric model and the rendered image is necessary to allow interaction directly on the rendered image. The paper is organized as follows.Section2deals with some basic terms and definitions which are necessary to explain the concept this paper is about.The fol-lowing Section3gives an insight into line-drawings as a medium of expression.Related work in thefield of computer generated line-drawings is examined.Sec-tion4explains the concept of“Drawing Resources”. This concept is used to control the level of detail in drawings as we show in Section5.Here we also show results of our work.The paper ends with some conclu-sions and some visions for future work.2Basic Terms and DefinitionsLine-drawings are pictorial representations in which the depicted objects of the reality are visualized exclusively with lines.This rather inaccurate definition serves the purpose at hand.It says that(1)line drawings are constructed with lines and(2)that due to this fact a line drawing is always an abstraction.However,this definition needs some further explanation.It is based on the term“line”,which we must deal withfirst. There are many ways to come up with a definition of the term“line”.Mathematicians do not use the term in general;instead they define several special types of lines like straight lines,parabolas,spline curves,etc. Their purpose is to have a tool giving the points a lineconsists of in an analytical way and thus to depict spa-tial relationships in2D or3D space.In our context,however,we do not need a definition by giving each single point.We need a general description of the term“line”and a possibility to decide whether a graphical object is a line or not.We are especially interested in describing all the lines one can draw man-ually.Looking closer at the drawing process in art reveals another way of defining a line.When drawing a line,a visible trace is left behind by a tool on a surface.The drawing tool(a pen,a brush or some sort of etching tool)is either capable of emitting a colored substance or of scratching the surface.The drawing surface(like paper,canvas or a copper plate)itself either takes on the color or is of such a consistency that it is possible to scratch it.Combining both,the mathematical and the artistic way of defining a line,we come to the following definition:A line is a curve which is the result of an un-interrupted movement of a drawing tool ontoa drawing surface and which is at least C1continuous.This definition covers all the mathematically defined lines,as long as there are two end-points(infinite lines would need an infinite movement of the drawing tool). If the movement is interrupted,the line ends.A line also ends in points without C1continuity.Thus poly-gons are divided into a set of single lines.A line never exists without its environment.A single line drawn in an area will never be regarded as a line alone.It is seen as inside of or in the foreground of aflat area.If one line already has relations to its environment there must be relations between multiple lines drawn on the same surface.With this relations one can construct complex objects which also relate to each other.All these relations create pictorial representations—“line-drawings”.In line-drawings,the shape of an object as well as light and shadows are expressed by the proper use of lines. Thus when looking at line-drawings,it is conspicuous that lines are used in different ways to obtain special effects.We therefore distinguish between •structure lines and•hatching lines.Structure lines depict the shape of an object.We regard as shape the contour as well as the internal structure. This leads to a furtherclassification:Figure1:Different line classes form a drawing.•Internal structure lines are used to show the inter-nal structure of an object,its composition,how it is constructed out of faces and sub-objects.•External structure lines form the contour of an ob-ject e.g.the object’s border to its environment. Hatching lines are used to show•the surface shape(is it aflat or curved surface etc.),•the structure of the object’s surface(material prop-erties),•lightning and•shadow casting.Hatching lines never appear alone.Their effect is caused by the“team-work”of many lines.Figure1shows the different kinds of lines mentioned above and how they together create a line drawing.3Line-Drawings as a Medium of ExpressionPictures are a medium of expression.This is a known fact and very often stated with the saying“A picture is worth a1,000words.”.N.Gershon states in[2]that the usual meaning is that one could pack more infor-mation into a picture than by using words to describe the same information.Images can even convey infor-mation that words cannot.On the other hand,words are sometimes more effective than pictures.An image of an unknown person driving a car gives the informa-tion“a woman driving a car”,but not“Ms.Smith is a good driver”.This can be done by using words in addition to or instead of the picture.Words can give definite information whereas pictures are in most cases somewhat fuzzy,sometimes they only give some general information.Due to this fuzziness pictures,like words,can be seen as an abstraction from reality.However,there is a wide range of pictures with different levels of abstraction.Thinking of photographs one can say that here the level of abstraction is very low in contrast to,for example,simple line-drawings.Ob-jects in a photograph can often be recognized imme-diately,whereas in a line-drawing the mental process involved in recognizing an object is more complicated and requires more“internal knowledge”by the viewer. This might be one of the reasons why the generation of pictures on a computer started with the goal to produce photorealistic images.A simple model of the photographic process serves as basis for the image cre-ation process,rendering.The knowledge used about the objects in a scene is limited to geometric informa-tion and environmental data(atmospheric data,light, etc.).Very seldom,more information,like object rela-tions(hierarchical structure of a scene,etc.)or infor-mation about the importance of an object in the scene is used.The last point might be obvious when talk-ing about photorealistic images since they are created using a model of a photographiccamera.Figure2:Schematic surgical drawing used in a textbook(taken from[3]).Line-drawings offer possibilities for a higher level of ab-straction.The fact that everything is expressed by the exclusive use of lines is one explanation for this.Look-ing closer at line-drawings,especially in thefield of sci-entific illustration,offers further insight.Here a painter can code abstract information into the picture by us-ing different techniques.With different line widths,for instance,the importance of an object in relation to the other scene objects can be emphasized.Some of these techniques are:•Show transparency of objects by using dotted or dashed lines.•Express the importance of objects by using dif-ferent dense hatching lines and different hatching styles.•Express different importance through different lev-els of detail.While an important object exhibitsa very detailed structure,unimportant ones aredrawn only with external structure lines.Because of these possibilities to show abstract informa-tion in pictures line-drawings are often used as illustra-tions.Examples are given in Figures2and3.Figure3:Line-drawing used in an architecture textbook(taken from[4]).While the use of line-drawings as illustrations is well known,the generation of line-drawings for illustra-tive purposes is a relatively newfield within computer graphics.A look at related work reveals the pioneer work of Saito and Takahashi[7]who created line-drawings based on3D models.They exploited G-buffers to enhance the resulting images,data structures which provide a link between the image and the under-lying model so as to allow interactive modification of the rendered image.Winkenbach and Salesin[9]mentioned the need for “visual abstraction”within computer graphics and used different hatching techniques to render more or less de-tailed objects.The system of Preim and Strothotte[5]is devoted to abstraction and focusing in line-drawings.This sys-tem works basically on a statistical analysis of the line-density within an image to decide which lines can be left out to focus on a part of an image.Simon Schofield s research in the area of non-photorealistic rendering,especially in thefield of new interaction techniques(see for example[6])had also valuable influence on our work.4“Drawing Resources”Computer generated line-drawings are generated based on a geometric model.This can either be a description of the drawing in2D or a3D model of the scene.In our research,we concentrate on the second way.The image is created out of a3D CAD model as the product of a rendering process.During this rendering process,the edges defined by the3D model are transformed into the structure lines of the drawing.In addition,we allow the user to add hatching lines.The resultant set of lines is what we consider to be“drawing resources”.Our concept is to measure the use of“drawing re-sources”for each object in a scene.Based on these measures,we can tell an object to use less resources and we give methods how this reduction should be per-formed.Strictly speaking,we try to characterize or to formalize the arrangement and the appearance of lines in computer generated line-drawings.There are several possibilities to measure the use of re-sources in line-drawings which reflect the real use more or less exactly.The most simple measure is to count the lines being used for the drawing of a specific object or of the whole scene.Another possibility is to determine the sum of the lengths of the lines used.These two measures can be combined in the average line length which is defined as the sum of line lengths divided by the number of lines.All these measures are absolute, they do not reflect the size of an object with respect to other objects.Rendered line-drawings are created from a3D model which consists of a set of edges.Taking this into con-sideration,it is also possible to specify the use of lines as the ratio between the number of edges in the model and the number of lines in the drawing.This measure can be seen as a clue of how good an object is visible. Two more values,which can be measured or obtained from the internal state of the drawing program,respec-tively,are line brightness and line width.Putting it all together the measures we use are:•overall line count:N=all lines1•overall line length:L=Ni=1length(line i)•average line length:l=LN•object visibility:N=Nall edges in model1•line width:to be measured•line brightness:to be measuredThese measures allow working directly on the drawing. There are neither modifications of the model nor inter-ventions in the rendering process itself.The3D model serves as a source for information about size and posi-tion of objects in3D space.The measures are a tool to control the use of resources in a drawing.How can they be applied to the ren-dering process?To answer this question,we studied line-drawings in scientific illustrations.Some of the techniques used here can be explained with our con-cept.The importance of an object,for instance,can be expressed by:•reducing the resources for all objects adjacent to the chosen object by drawing them with brighter lines,•drawing the object itself with either darker or wider lines or•drawing the object with all structural detail(sub-objects)and applying a very detailed hatching whereas the surrounding objects are drawn with less detail.For working with these measures,a structured descrip-tion of the drawing is necessary.Also relations between the lines in the drawing and the corresponding objects in the3D model are important and the more informa-tion of this kind is available,the better one can apply the concept of drawing resources.What is the additional information stored in the3D model used for?To draw an object with more or less detail,all lines belonging to this specific object are needed.If there is a data structure available which says the object consists of several sub-objects it is possible to choose a level in this hierarchy,the reduction process should be applied to.It is also possible to change the level of detail just by removing sub-objects,leading toa less structured drawing and thus to a higher level of abstraction.Information out of the3D model can be used tofind the external structure lines of an object or to get the topologically adjacent objects.A way for automatic reduction of drawing resources is to get the distance of an object to the viewer and to draw the objects with an amount of resources which depends on this distance. How,for example,such an automatic processing of a drawing can be done is the topic of the next section. 5Implementation and ResultsThe basis for our prototypic system is the“sketch-renderer”as presented in[8].This renderer creates a set of2D lines out of a3D model.We use standard geometric models with additional information about an object structure in the scene.That means parts of the geometry are grouped into objects.These object def-initions can be nested.The resultant object structure is preserved in the rendering process to build a tree-like representation,which contains pointers to the ob-jects of the scene with all lines in the image belonging to them.The renderer itself only produces structure lines.The resulting pictures can be extended by creat-ing hatchinglines.Figure4:The system consists of two parts with user interaction taking place in both of them.Generating a special line-drawing with the system is a two stage process(see Figure4)where user interaction takes place in both stages.Thefirst stage consists of generating the line-drawing from a3D model by means of the renderer.This basic drawing contains only struc-ture lines.A user can add hatching lines,which are available in three“styles”:•normal lines having a specified width and bright-ness,•lines having different width at the end points(the effect is like using a felt pen with different pressure) and•lines having different brightness at the end points (this effect is like using a pencil with different pres-sure).Figure5shows an image after thefirst stage of the image creating process.It just uses different hatching lines,there were no modifications of the drawing re-sources done at thispoint.Figure5:An image of a nutcracker where different hatching lines are used.The existing image is the basis for all further modifica-tions done in the second stage.Here a user can change the level of abstraction an object is depicted with.This happens in three different ways(see Figure6):1.All attributes, e.g.line width,line brightness,hatch density,etc.are manually changed and ob-jects which should not appear in the scene are deleted.2.A user selects an object and specifies what levelof detail he wants the object to be drawn with.The system generates a proposal by changing the attributes in a rule-based manner.3.The system generates a proposal for the wholescene.This is done by inspecting the distance of an object from the viewer’s(camera’s)position.Figure6:The limitation of drawing resources happens in three different modes which can be combined.The object nearest to the viewer is drawn with all detail available.The level of detail decreases with increasing distance of the object from the viewer. All these proposals can be refined by a user while in-teracting with the graphics in the way described above. The three interaction modes can be combined to pro-duce the imagefitting best the user’sideas.Figure7:A picture of a boom created with our system.The weights are of importanceTo describe the work with the system the image in Fig-ure7is used as an example.It has been produced in the following way:1.The3D model is rendered with the sketch-renderer.As a result we get all structure lines of the image.2.A user adds hatching lines.This is done by select-ing an object and adjusting the hatch parameters like:•hatch density,•direction of hatching lines,•“style”of hatching lines(normal,differentbrightness,different line width).3.The user’s intention is to emphasize the weightsin the picture.This can be done differently.The chosen method here is:(a)Reduce the resources for the object“boom”e brighter lines and reducethe hatch density.(b)Draw the weights with wider structure lines. Another example where the boom itself is emphasized is shown in Figure8.Figure8:The same model as in Figure7with a different re-source distribution.The boom itself is emphasized. Since the model is hierarchically structured it is easy to specify the reduction values and methods for an ob-ject.The chosen method and the values are recursively applied to all sub-objects in the tree-structure.The model structure is also used for interaction.The selection of a model part is done by“hierarchical pick-ing”.This means an object is selected by picking with a pointing device(mouse)in the drawing.The selection is shown by drawing marks at the corners of the bound-ing rectangle.Thefirst pick selects the highest object in the hierarchy the picked point belongs to.Every fur-ther pick goes down one level in the model hierarchy. So a user selects along a path like for instance“boom→vertical beam→front face”.The selection path goes down till the level of single polygons is reached.The implementation was done on Silicon Graphics Workstations using C++and GL for the graphical out-put.The algorithms standing behind the rules which are applied as reaction of a user’s input are of differ-ent complexity.Because of an internal tree-like repre-sentation of the geometric model most operations,likechanging the line brightness or width,can be done very fast by traversing a sub-tree.Operations,which cannot be performed this way,like changing the hatch density, need more time.Here we have to compute new hatch-ing lines to preserve an equal distance of the lines.In the current implementation,we restrict ourselves to polygonal CAD models.Best results we got using ar-chitecture models.Out hatching techniques work best with models having not to much regular faces.6ConclusionThe method and the tool we present helps a user to cre-ate a line-drawingfitting his needs.The system works with standard CAD models and is therefore widely ap-plicable.With the use of different widths and brightness of lines, it is possible to express more than only different levels of detail.With lines of different brightness,the effect of depth-cueing can be simulated.Lines with differ-ent width at the end and the beginning can depict the surface shape of an object.Our system does not use free form surfaces.The use of splines and free form surfaces might increase the num-ber of models we can handle as well as the possibilities for reducing the drawing resources.We left this exam-ination for our future work.The support for the system coming from the model and from the underlying renderer is very important.Having more references from the2D objects to he3D model yields even more opportunities.Some of them which will be valuable to examine are:•The use of topological information from the3D model.So,for example,if an object should be emphasized,all adjacent objects could be drawn with slightly less resources.•If there are references from a(structure)line in the drawing to the corresponding edge in the model the classification in internal and external structure lines can be done easily.When distinguishing be-tween internal and external structure lines,one can emphasize an object by drawing it with a detailed structure whereas all remaining objects are drawn just with their external structure lines.To enter the hatching or reduction parameters,we cur-rently use an OSF/Motif interface.A new method of interaction,we will study in future,is to use the point-ing device as a kind of eraser,removing more and more detail when moving over a surface.Here we need a de-tailed structure of the model to get information about the object the mouse is pointing to.This concept should lead to a very powerful interaction technique, however,a very effective way to do the rendering in near real-time as reaction to the interactions is needed. The system presented can be used for a wide variety of tasks.First of all the resulting images serve as illus-trations in publications.As it is to be seen in Figure7 and Figure8a user has the possibility to emphasize different aspects of a scene for instance for comparison. The created images can be exported either as bitmap or PostScript so that they can be used in any word pro-cessing system.A second—even not intended—area of application is arts,since the resulting images have a certain appeal which might be applicable for decora-tive purposes.See for example Figure5which shows an image created for a Christmas greeting card. AcknowledgmentsWe wish to thank all our colleagues in Magdeburg for fruitful discussion,especially Andreas Raab who imple-mented the sketch-renderer and Axel Hoppe who did all the modeling.Thanks also to Ute Lau,who proofread the manuscript.References[1]J.D.Foley,A.van Dam,S.K.Feiner,J.F.Hughes:Computer Graphics.Principle andPractice.Addison Wesley Publishing Company,Reading,MA,1990.[2]N.Gershon:“The Power and the Frailty of Im-ages.Is a Picture Worth a Thousand Words?”In:Computer Graphics,Vol.:29,No.:4,1995,pp.35–36[3]E.R.S.Hodges:The Guild Handbook of Sci-entific Illustration.Van Nostard Reinhold,NewYork,1989.[4]W.Koch:Baustilkunde.Mosaik-Verlag GmbH,M¨u nchen,1991.[5]B.Preim,Th.Strothotte:“Tuning renderedline-drawings”.In:Proceedings of WSCG’95,(Plzeˇn,February,14—18)1995,pp.228–238. [6]nsdown,S.Schofield:“Expressive Ren-dering:A Review of Nonphotorealistic Tech-niques”.In:IEEE Computer Graphics and Ap-plications,Vol.:15,No.:3,May1995,pp.29—37.[7]T.Saito,T.Takahashi:“ComprehensibleRendering of3-D shapes”.Proceedings of SIG-GRAPH’90(Dallas,Texas,August6—10).In:Computer Graphics,Vol.:24,No.:4,1990,pp.197–206.[8]Th.Strothotte, B.Preim, A.Raab,J.Schumann, D.R.Forsey:“How to RenderFrames and Influence People”.Proceedings of Eu-rographics1994,(Oslo,September12—16,1994).In:Computer Graphics Forum,Vol.:13,No.:3,1994,pp.455–466.[9]G.Winkenbach,H.Salesin:“Computer gen-erated Pen-and-Ink-Illustration”.Proceedings ofSIGGRAPH’94,(Orlando,Florida,July24—29).In:Computer Graphics,Vol.:28,No.:4,1994,pp.91–100.Citation information:Stefan Schlechtweg,Thomas Strothotte:Rendering Line-Drawings with Limited Re-sources.In:Proceedings of GRAPHICON’96(St.Peters-burg,Russia,July1–5,1996),vol.2,pp.131–137,1996。