Interface, Design and Visual Indexing

Each of the panel are working in related ways in the context of this session, to address the storage and retrieval of the stories of a modern oral and visual culture. Four distinct projects will open out the approaches and thinking being pursued and the overlap that exists between them. We have become aware of one another’s work over the last nine months, have been working on our separate projects for varying periods with and without budgets, and also have in common development cycles of from 5 – 10 years. These short presentations will each highlight the specific problem encountered or theoretical concept being tested and why the outcome of the project could be of wider social value.
KEYWORDS: interface, indexing, interactive,
hypermedia.
PAPER
Whatever the origin of our representations…..they
must all, as modifications of the mind, belong to the
inner sense. Kant
Introduction
Descriptions of Human Computer Interfaces rely
heavily on visual metaphors developed in the
mechanical machine age – the printed page, the
desktop, the map, the graph paper, the soundtrack
dubbing cue sheet…. These have provided models
towards HCIs which in the transitional sense
complement the workings of human memory but
which largely fail to stimulate the individual users
imagination upon becoming ‘immersed in
knowledge’, or other risks associated with the
computer-mediated experience.
Random access in the computer as opposed to the
book, is capable of extending the usefulness of the
written word and delivering, on-demand, the spoken
word, sound and picture. Though the means of
achieving this is currently in technical development
the general purpose interface capable of delivering
indexed sound and image beyond the notion of ‘the
thumbnail’ image set in a key word context, is an area
in need of further research. Whilst resources have
been invested in the imaging of data for systems
management, (in finance, security, aviation etc), little
has been expended in the expansion of tools for the
humanities and the development of human
consciousness that appeal to the intuitive mind or the
curious spirit.
Whilst the coding options within the literate society has
recently undergone rapid extension – email, hypertext,
server messaging, SMS - the rediscovery or reinvention of
an interactive oral culture using text, graphics, moving
images and sound delivered over high-speed networks
will be unable to produce similarly fundamental outcomes
without access to indexing that avoids the use of one
coding system – text – in order to access another – images
and sound. Whilst we acknowledge the impact of
computer-mediated gaming, the financial imperatives that
have driven this development has eclipsed the possibility
of advancing in areas unrelated to entertainment.
We will each speak to our printed position statements for
5-10 minutes, as the focus will be on comparing each of
the models we are working with in order to clarify the
ways forward, during a concluding discussion, in the
more universal adoption of the computer-mediated
presentation and reception of knowledge and experience.
Visual Indexing and the Language of Gesture
Chris Bowman:
Since the invention of cinematography motion pictures
have undergone continual advances in both technology
and content. Now, with the introduction of interactive
technologies, the conventions of storytelling through
motion pictures are being challenged and reshaped.
Primarily, a place is being constructed for the viewer to
'enter' the story and effect change, making choices and
seeing the consequences in real time. Given the basic
limitations of the computer as auditor, the creative effort
of interactive content developers, designers and artists
becomes focussed on how to build worthwhile options
and how to fashion sympathetic responses. Visual
indexing in this context works to move beyond the
dominant theory of film narrative to explore intuitive
storytelling in terms of mediation, gameplay and
meaning.
Currently, I am exploring the rich terrain of visual
indexing through the Orpheus Project. Orpheus, a work in
progress, will be a modern film interpretation of the
immortal story told through the symbolic artistry of
Sankai Juku, one of the world's most outstanding dance
ensembles. In the world of interactive possibilities
Orpheus proposes to use a three part system of navigation
to enable the viewer to enter the story. I propose that the
three part system will consist of discrete forms of visual
indexing to facilitate viewer participation in the following
manner:
MelbourneDAC 2003
1) Polar Keys – present the viewer with the
opportunity to create changes, sometimes harmonious,
sometimes abrasive to the story. This is achieved by
providing the viewer access to alternative sequences
that explore contrasting subplots and metaphors.
These alternatives influence the events throughout the
story and most significantly, they are designed to
affect the motivational influences of the main
characters.
2) Portals – an opportunity to use modes of interaction
which explore the hidden architecture of the film and
graphic content. In particular, I wish the viewer to
manipulate the speed, depth of field, and layering of
audio/ visual content. The consequences of this
manipulation is to allowing the audience to experience
shifts in the key events in the story and experience the
sequences from another character's point of view.
3) Narrative Gates – fashion an opportunity to engage
in elaborate passages of text (the Sonnets to Orpheus
by Rainer Maria Rilke and the 3rd Century AD Orphic
Hymns) and image accompanied with chorus/voice
over. These resonant passages create shifts in the story
which embody the spiritual relationship between
nature, humanity and the universe. The viewer will
explore these passages as sound and film
compositions.
Much of my work here is focused on the visual codes
and index systems used to build and shape this three
part system of navigation. In particular, I am
interested in further developing the mechanics of
computer interaction to enable the audience to remain
immersed in the story and its elaborations. From this
comes my conviction that the language of manual
gesture is the most promising way to proceed. The
dimensionality of simple hand movements may
tracked and these are translated into corresponding
visual and audio transformations.
With the generous support of the Australian Film
Commission and the Australia Council the Orpheus
project has moved through progressive phases of
script development for film, DVD formats and gallery
installation. At present, I am developing a prototype
for the gallery installation using a Narrative Gate as
content. This prototype will test the visual indexing of
a 'set' of immersive narrative experiences as they are
mediated through a 3D configured Virtual Reality
stereographic interface and 'cinematic' gallery
environment. This technical configuration and the
visual indexes are designed to offer a compelling and
absorbing sensory environment within which viewers
may interactively weave their own pathways to
explore the complexity of themes, characters and
settings. In this manner Orpheus will unfold before
them in a way of their choosing, which is both
personal and unique.
Orpheus is aimed at a broad audience which includes
not only those most interested in theatre, dance and
music but also those who appreciate the meld of art and
technology. It is intended that the convergence of the 3D
configured Virtual Reality stereographic interface and
cinematic content will, in part, lead towards networked
systems of communication in the fields of collaborative
medical/biotechnologies, nanotechnologies, art and
design.
Envisioning Corangamite
Jacqueline Gothe and Daniel Ireland
The process of interface design and related notion of
indexing find their form in this project in complex ways.
The understanding of indexing through this project
reveals itself, as a social phenomena, with an immutable
relationship between the real world/outside/corporeal and
the virtual world/inside/screen. In this scenario the
capability of providing an interactive consultation
function within a community and policy development
framework where the stakeholders range across a diverse
user profile provides a site of investigation.
The project team are interested in raising two areas of
indexing that is evident in the interface design. First is the
issue of the representation of consultation and how that
arrives in this project. Second is the indexing that enables
an immersive textual experience for the engaged user with
an ease of access to the underlying arguments,
information and data that support the strategic document.
Sustainable use of natural resources is a major task facing
Australia. It is complex and contested ground, where
communication between participants - scientists, technical
advisers, resource managers, landholders and
environmental groups – is central to reaching agreement
on problems, causes, goals, priorities and actions.
The project Envisioning Corangamite
(http://www.ccma.vic.gov.au/rcs ) – supporting
knowledge sharing through information design in the
planning for sustainability across real and virtual spacestakes
as its starting point the research, design and testing
of knowledge objects and interactions in planning for
natural resource management at regional and local levels
of scale. The issues of interface design and indexing are
major components in the focus of the project. The project
is currently in its third iteration of Stage 1. It is hoped that
the Project can move into Stage 2 and 3 over the next four
years. These next stages will provide the opportunity to
explore representations of the environment in a web, print
and temporal based forms.
This project is based on the communication strategy
Exploring the Communication Landscape (see
http://www.educ.dab.uts.edu.au/ccma/ccma.swf ) for the
Corangamite Catchment Management Authority for the
Corangamite region – an area of 13,340 square kilometres
in southwest Victoria stretching from Geelong along the
Great Ocean Road, north through Cobden and
Camperdown, Lake Corangamite and part of Ballarat. The
MelbourneDAC 2003
region is defined by the aggregation of its four river
basins – Moorabool, Barwon, Lake Corangamite and
Otway Coast, plus three nautical miles out to sea.
The four strategic approaches are
1. Broaden models of communication to include an
interactionist perspective where the desired outcome is
shared meaning.
2. Design with an understanding of complexity
utilising principles of information design and visual
processing.
The indexing system that has emerged as a
representation of the consultation and its contribution
to the interface design is a direct result of the
articulation by the team that developed the Regional
Catchment Strategy of a commitment to community
empowerment in the process of natural resource
management and the determination that this would be
an ongoing process over the five year renewal. This
attitude was evident in the notion of the question and
the comment which becomes a major index to develop
the process of knowledge sharing in the region.
The second determiner of index and interface was a
commitment to the text as the foundational. The
Regional Catchment Strategy 2002-2007 is not one
document but rather a suite of documents comprising
a summary, a strategic overview, fourteen published
reports, supporting strategies and a record of the
regional discussion. This complexity is then placed in
the even more complex context of State and Federal
natural resource management strategies and policy
papers.
Maintaining integrity to the breadth and the ongoing
nature of the process demanded an acceptance of the
browsing nature of information gathering on the web
and the impossibility of easily grasping the whole. The
impetus to find simplicity was counter-pointed by an
understanding of the inability to mobilise reduction as
a satisfactory strategy in this context of what appears
to be vast amounts of information. However the depth
of relationships and the linked arguments across
policy documents was a major index. The design
strategy is evident in the linked reports, comments and
hyperlinks to other relevant websites. It is this textual
indexing that provides for an immersive non-linear
experience.
The aim, for the Corangamite regional Catchment
Strategy 2002-2007, in its real and virtual
manifestations is to facilitate the ongoing interaction
that results from the comments and questions, the
updated research in the form of scientific data, reports
and relevant strategies and regional news which will
be available on the web site, in addition to ongoing
live discussion in real spaces. The website becomes a
focus to manage this content so that in 2007 when this
Strategy is renewed the evidence of the conversation will
be maintained in the database supporting this site.
PathScape
Mike Leggett:
The PathScape project seeks an indexing system capable
of accessing personal and public digital documents placed
into a topographical context.
There exist several software tools (such as ArcView),
related to topography and recorded time and place. These
are widely used in the industries related to so-called
environmental planning – water and land management,
urban layout, national parks, mining and agriculture, etc.
They are ingenious, specialised tool sets based on data
derived from scientific method - measurement. Combined
with GIS satellite data and a range of plug-ins that enable
digital images, sound and text files to be attached to
specific coordinates, this allows extensive profiles of
geographical locations to be constructed and navigated in
real-time. The Humanities have adapted these tools,
archaeologists and social scientists most notably, with for
instance, TimeMap that links through a combination of
text and map metaphor, personal oral histories with
localities.
My problem with such tools is the plethora of styles and
codes they usually incorporate, using maps, diagrams,
graphical and typographic devices , each inflected with
current tools and fashions in interface design. The user’s
encounter is like a visit to the aquarium, gazing through
the glass at other peoples’ lives, before moving onto the
next container.
The PathScape project sets out to test the tide line
between the practicalities of delirious immersion and the
possibilities of indexing the cogent experience, as lived
by the subject and lived by others. In this sense it can
become a tool, like for instance those used by amateur
genealogists, for the accumulation of a library of personal
audio and visual material. It is related to the notion of
‘infovis’ as described by the mathematician Tamara
Munzner as being “about tools that exploit the human
visual system to help people explore or explain data.
Interacting with a carefully designed visual representation
of data can help us form mental models that let us
perform specific tasks more effectively.”
PathScape is an interactive multimedia project
progressing through several stages and adopting several
iterative forms. In the completed prototype audio-visual
indexing forms operate within an interface design
developed with a small team of sound and visual artists.
The project accesses representations of the natural world
through a combination of gesture and iconic image
fragment. The outcome of a recent iterative version,
GreenScape, the interactive encounter occurs in a public
gallery-type space utilizing a screen and sound
MelbourneDAC 2003
deployment which in conjunction with a gesture
recognition system that goes beyond mouse-scale
movement, will expand the meaning of a journey into
the realm of the performance, as each user in turn is
observed talking a walk through the landscape.
The appeal will be to the users knowledge and
experience of the natural world, however wide or
narrow, sympathetic or antipathetic that might be, and
its remediation as an artificial topography.
Encountering this range of immersive states will
engender a sense of a favoured space. In moving from
A to B, or backwards and forwards in parts of that
track in exploration, the user will learn the pathway as
a visual indexing system through the visual cues (loci)
that lead back to the combination of files that deliver
the most pleasure, meaning, or other stimulations. The
users cultural preference and temperament determines
this, whilst giving exposure to, (also as an observer),
the cross-cultural preferences of others. The Japanese
concept of ma for instance, connotes the complex
network of relationships between people and objects.
This notion of space-time continuum is distinct from
the neutrality assumed of post-Renaissance space,
which is then personally and collectively colonised.
The project seeks to reach both general and specialist
audiences and can be adapted to do so in disc,
installation (interactive cinema) or on-line format.
Media assets can be added or removed according to
the installational need quite easily, as the engine will
review folder contents at each reboot – the framework
is dynamic in this sense and can be regarded as a tool.
The project accesses representations of the natural
world through a combination of gesture and iconic
image fragment, so that as Nikos Papastergiadis has
put it: “..I seek to grasp the sense of place that is
created as art stimulates sensations and engages
relations with other people.”