Thoughts on an Image Description Vocabulary

Version 1.1
31st August 2003.
Previous version
Jim Ley, Jibbering.com

Current Image vocabularies

A quick review of what's out there currently.

My SVG image vocab - This vocab is used in my SVG Image annotator, it contains a Polygon class, which has a property which uses the SVG path element as syntax to define the region. It also contains compass direction points for specifying where in the image the elements are.

Masahide Kanzaki's vocab - This vocab has an Area class, which has a shape property, and coords properties, the shape uses poly, circle, rect etc. from HTML imagemaps, and the coords property to define the co-ordinates.

Chris Goad has an RDF geometry vocab which he's designed primarily for mapping applications, but in defining geometry could be used for the regions. There's also SVG Linearization and Accessibility from Ivan Herman and Daniel Dardailler which is focussed towards describing SVG images for accessibility reasons, it does contain useful predicates for defining the relationships between subparts of the images.

Proposed Schema

Area class

This is a class which defines an area of an image.

 <rdfs:Class rdf:about="http://jibbering.com/vocabs/image/#Area"
  vs:term_status="testing" rdfs:label="Area" rdfs:comment="An Area of an image.">
  <rdfs:isDefinedBy rdf:resource="http://jibbering.com/vocabs/image/"/>
 </rdfs:Class>

Polygon class

A subclass of Area, which more says that the Area is a polygon, and the points property will be in a format which defines a polygon, i.e. a series of points.

  <rdfs:Class rdf:about="http://jibbering.com/vocabs/image/#Polygon"
  vs:term_status="testing" rdfs:label="Polygon"
  rdfs:comment="An area which is the shape of a Polygon.">
  <rdfs:subClassOf rdf:resource="http://jibbering.com/vocabs/image/#Area"/>
  <rdfs:isDefinedBy rdf:resource="http://jibbering.com/vocabs/image/"/>
 </rdfs:Class>

Rectangle class

A subclass of Area, which more says that the Area is a rectangle, and the points property will be in a format which defines a rectangle, i.e. a set of two points.

  <rdfs:Class rdf:about="http://jibbering.com/vocabs/image/#Rectangle"
  vs:term_status="testing" rdfs:label="Rectangle"
  rdfs:comment="An area which is the shape of a Rectangle.">
  <rdfs:subClassOf rdf:resource="http://jibbering.com/vocabs/image/#Area"/>
  <rdfs:isDefinedBy rdf:resource="http://jibbering.com/vocabs/image/"/>
 </rdfs:Class>

Circle class

A subclass of Area, which more says that the Area is a circle, and the points property will be in a format which defines a circle i.e. an x,y and a radius

 <rdfs:Class rdf:about="http://jibbering.com/vocabs/image/#Circle"
  vs:term_status="testing" rdfs:label="Circle"
  rdfs:comment="An area which is the shape of a Circle.">
  <rdfs:subClassOf rdf:resource="http://jibbering.com/vocabs/image/#Area"/>
  <rdfs:isDefinedBy rdf:resource="http://jibbering.com/vocabs/image/"/>
 </rdfs:Class>

points property

The Points property is used to define the actual points used to describe the shape, a list of x,y points for a polygon, two points for a rectangle defining the corners, a center and a radius for a circle. For a polyline path, the syntax is the same as the points property of the SVG polyline element. For a rectangle it is the two corner points, and for a circle it is three numbers x,y and r. The points are seperated by whitespace.

The points are defined to be in the co-ordinate space of the image, which for raster images can be pixels, if the image is not raster, but vector, then it will be dependant on the height and width properties.

 <rdf:Property rdf:about="http://jibbering.com/vocabs/image/"
  vs:term_status="testing" rdfs:label="points"
  rdfs:comment="The points which define an Area.">
  <rdfs:domain rdf:resource="http://jibbering.com/vocabs/image/#Area"/>
  <rdfs:range rdf:resource="http://www.w3.org/2000/01/rdf-schema#Literal"/>
  <rdfs:isDefinedBy rdf:resource="http://jibbering.com/vocabs/image/"/>
 </rdf:Property>

height and width properties

These are the height and width of the image

 <rdf:Property rdf:about="http://jibbering.com/vocabs/image/#width"
  vs:term_status="testing" rdfs:label="width"
  rdfs:comment="The width of an image or Area in pixels for raster elements.">
  <rdfs:domain rdf:resource="http://www.w3.org/2000/01/rdf-schema#Resource"/>
  <rdfs:range rdf:resource="http://www.w3.org/2000/01/rdf-schema#Literal"/>
  <rdfs:isDefinedBy rdf:resource="http://jibbering.com/vocabs/image/"/>
 </rdf:Property>

 <rdf:Property rdf:about="http://jibbering.com/vocabs/image/#height"
  vs:term_status="testing" rdfs:label="height" rdfs:comment="The height of an image or Area .">
  <rdfs:domain rdf:resource="http://www.w3.org/2000/01/rdf-schema#Resource"/>
  <rdfs:range rdf:resource="http://www.w3.org/2000/01/rdf-schema#Literal"/>
  <rdfs:isDefinedBy rdf:resource="http://jibbering.com/vocabs/image/"/>
 </rdf:Property>

area property

These are the area of the image or region, this is to allow queries on the RDF without having to teach the RDF tools how to calculate the areas from the points property.

 <rdf:Property rdf:about="http://jibbering.com/vocabs/image/#area"
  vs:term_status="testing"
  rdfs:label="area" rdfs:comment="The area of an image or Area.">
  <rdfs:domain rdf:resource="http://www.w3.org/2000/01/rdf-schema#Resource"/>
  <rdfs:range rdf:resource="http://www.w3.org/2000/01/rdf-schema#Literal"/>
  <rdfs:isDefinedBy rdf:resource="http://jibbering.com/vocabs/image/"/>
 </rdf:Property>

fraction property

This is the fraction of the whole image that the Area takes up.

 <rdf:Property rdf:about="http://jibbering.com/vocabs/image/#fraction"
  vs:term_status="testing" rdfs:label="fraction"
  rdfs:comment="The fraction of the Area to the whole image.">
  <rdfs:domain rdf:resource="http://www.w3.org/2000/01/rdf-schema#Resource"/>
  <rdfs:range rdf:resource="http://www.w3.org/2000/01/rdf-schema#Literal"/>
  <rdfs:isDefinedBy rdf:resource="http://jibbering.com/vocabs/image/"/>
 </rdf:Property>

hasPart property

This property relates an Image, with an Area.

 <rdf:Property rdf:about="http://jibbering.com/vocabs/image/#hasPart"
  vs:term_status="testing" rdfs:label="hasPart"
  rdfs:comment="The image in the resource has a part..">
  <rdfs:domain rdf:resource="http://www.w3.org/2000/01/rdf-schema#Resource"/>
  <rdfs:range rdf:resource="http://jibbering.com/vocabs/image/#Area"/>
  <rdfs:isDefinedBy rdf:resource="http://jibbering.com/vocabs/image/"/>
 </rdf:Property>

Open Issues

There's no property for relating an Area to thing, Previously I used img:depiction for this, however foaf:depiction is suitably defined, although it means that the Area class would have to be a subProperty or equivalent to foaf:Image, which may not be appropriate, certainly needs some thinking on.

Rather than modelling the shapes as subClasses there could be a property shape to indicate the shape, this certainly needs looking at.

There's nothing in it to define the location of the elements within the image, this is something I very much want to get in, but need some more research, especially on the sort of positioning which is useful. My compass points from the previous version were proved to be unsatisfactory in some testing Dave Pawson kindly provided.

Acknowledgements

Thanks to Masahide Kanzaki (神崎正英) for initial discussion at unifying our vocabularies