This document reports on a research in progress I am conducting during my final year at the University of Siena (Italy), Department of Communication Sciences; the research is supervised by Prof. Peppino Ortoleva (course of Theory and Techniques of New Media).
In general, multimedia hypertext is a medium in which the contents are entrusted mostly to text and images, whereas the music is often used only to create an agreeable background. The goal of this research is to examine all the possible uses of music in a hypertext to implement a new conception of multimedia language, so as to make music a real carrier of information and knowledge, at the same level of textual and visual codes.
Starting from the analysis of a sample of the existing multimedia hypertexts, a typology will be defined to figure out several models and functions of hypertextuality:
search needs (this is the prevalent logic of open on-line hypertexts as the Internet)
playful exploration (for instance, in interactive games)
functions deriving from audio-visual tradition (soundtrack as an element giving the images a structural coherence)
specific functions in those hypertexts where the structure itself expresses the author's creative thought (e.g. artistic or narrative fiction hypertexts, hypertextual essays)
According to the hypertextual model at stake, the use of music will be different.
In relation to this reasoning on hypertext, there is that one about multimedia. According to the state of the art of available production, we can give this word several meanings: it can be defined as a system to connect different languages as text, images, sound (author level); or a system to connect different senses (reader level); or a system to organize a great deal of heterogeneous materials (e.g. multimedia hypertexts that are simple data-bases); or as a mean to attract attention and interest.
Also in this case, the various ways to intend multimedia strongly affect the use of music, which is a medium among the other media and at the same time a sensorial experience, in other words a carrier of information and emotion.
In general, except the case of a musical hypertext (e.g. a hypertext about Beethoven or he Aerosmith), the music is not much more than a background, a resonant wall paper for a pleasant navigation. But, studying in depth the rhetoric of the hypertext, different and more profound functions can be found at three levels:
semiotic level: music works as a metalanguage and gives the product an identity as well as the graphic environment; or it is a way to establish a pact with the reader (in the most trivial sense, it is a signal declaring "this is a multimedia product"). Choosing a certain type of music instead of another one in a determined section of the hypertext is a way to suggest a possible interpretation for the text and images it is combined with. In relation to the kind of hypertext in question (more or less rigid, more or less hierarchic), the music can underline or adapt to the specific discursive structure in action.
syntactic level: music becomes an element of spatialization, for it generates resonant environments. Navigation in hypertext is a discontinuous shift between these environments (from one to another); the reader, by his virtual movement, creates a syntax while the text, thanks to its flexibility, adapts to his information needs. This fact raises the problem of how to set an underlying coherence and how to face the reader's challenge to the unity of hypertext. If the dialectic of links is entrusted to music, then at stake there is a new conception of hypertext because music becomes also a model to build its structure, and we shall need new metaphors to define its shape and use. This section of the research is also related to film editing theories, in particular to the contribution of S. Ejsenstein, which anticipated the creation of a language where different media are thoroughly integrated and have the same structural relevance.
pragmatic level: music can serve as a guide for the "wanderer" and enable him to do an oriented navigation; this is useful in a hypertext with a great deal of links at all levels. That means, for instance, using music so as to point out similarities or oppositions between different sections of the hypertext; in this way, the reader can draw in his mind a semantic map of the hypertext not only related to text contents. Moreover, music and sound can be exploited to enrich and improve the interface (in this section there will be also an analysis of how to do a good use of acoustic icons).
As it is evident, the three levels partially intersect, so what is said at one level is also relevant for the other two.
As an application of the research, there is in progress an experimentation for the construction of a music-based hypertext by testing the compatibility between some musical forms and different kinds of hypertexts ("pure", hierarchical, semi-hierarchical).
