Interactive Systems
Contents
- Overview
- Interactive Applications Archtecture
- A DSL for Interactive Applications
- Implementation
- Syntax Definition
- Semantic Definition
- Handling Events
- Simulation
- XML Representation
for Applications
Overview
An increasing number of interactive applications can be downloaded onto devices such as mobile phones, PDAs, web-browsers and TV set-top boxes. The applications involve presenting the user with information, options, menus and buttons. The user typically enters information by typing text and choosing amongst alternatives. An event is generated by the user clicking a button or selecting from a menu. Once an event is generated an engine that services the interactive application processes the event, updates its internal state and then produces a new dialog to present to the user.The dialogs required by the interactive applications tend to be fairly simple and are often used in conjunction with other applications such as being broadcast together with TV video and audio content. The technology used to construct the interactive application should be accessible to as broad a spectrum of users as possible, including users whose primary skill is not in developing interactive software applications.
Technologies used for interactive displays include Java-based technologies such as the Multimedia Home Platform (MHP) , HTML and JavaScript. These technologies are very platform specific. They include a great deal of technical detail and are certainly not approachable by a non-specialist. Furthermore, the general low-level nature of the technologies does not enforce any standard look-and-feel to interactive applications. The applications developed for a given client (for example a single TV company) should have a common look and feel that is difficult to enforce at such a low-level.
A common way to abstract from technical detail and to enforce a common look-and-feel for a suite of applications it to develop a domain-specific language (DSL) whose concepts match the expectations and skill-levels of the intended users. A DSL for interactive applications will include constructs for expressing textual content, buttons and choices. The DSL leaves the rendering of the display features and the event generation to a display engine that enforces a given look-and-feel. The display engine can be replaced, changing the look-and-feel without changing the DSL.
In addition to the DSL supporting high-level features for expressing display content, it must provide some means for describing what the application does. Normally, application processing algorithms are expressed at a low-level in program-code. If the DSL is designed with an execution engine then the same approach to abstraction from rendering detail can be applied to abstraction from the operational detail.
An executable DSL is a language for expressing complete applications without detailed knowledge of implementation technologies. The xDSL is a modelling language, instances of which are expressed as data. An execution engine processes the data and runs the application. The xDSL engine can be embedded within devices and other software applications in order to run the models.
This paper describes the design of an xDSL for interactive applications. The xDSL has a textual syntax and is implemented in the tool XMF . XMF is a platform for developing DSL-based applications. It provides an extensive high-level language called XOCL for developing applications and XOCL can be extended with new language constructs. The design of XMF has been based on Common Lisp , Smalltalk, Scheme and ObjVLisp . The DSL for interactive applications described in this paper can be developed on any platform, but XMF is ideally suited to the task. XMF will be available in the near future as an open source edition from Ceteva Ltd.
The rest of this paper is structured as follows: section describes an architecture for interactive applications based on an xDSL; section describes the xDSL from the point of view of the application developer, it outlines the language features in terms of a simple application involving a quiz; section describes how the xDSL is implemented on XMF; section shows how the rendering engine can be simulated and connected to the xDSL engine to simulate the execution of an interactive application; section shows how the application models can be serialized as XML; section concludes by reviewing the key features of the approach and the technology used to develop the xDSL engine.
Interactive Application Architecture
Figure shows an overview of the architecture for an interactive application xDSL. XMF is used as the DSL implementation engine. XMF provides facilities for developing text-based modelling languages and their associated execution engines. The xDSL is written as a model in XMF, the applications are then loaded onto the xDSL engine and executed.
The application generates display information in a general-purpose format; in this case XML. The XML display information is sent to a rendering engine. The engine understands the display features of the DSL and interprets them in terms of the rendering technology. This achieves a separation of concerns whereby the DSL can focus on the information content and execution logic whereas the rendering engine can focus on a standard way of displaying the information without necessarily having to understand anything about the application that is being rendered.
The rendering engine controls the hardware that interacts with the user. The user generates events that are sent back to the xDSL engine. In principle the events can be very detailed and can be encoded in a suitable DSL. The example presented in this paper uses a very simple encoding of events.
When the xDSL receives an event, it must process the data in an appropriate way to produce more display information for the rendering engine. The processing information is expressed in the application model running on the xDSL engine. The display/event loop continues until the application is terminated.
The architecture shown in figure has been used a number of times based on the XMF processing engine. The development environment XMF-Mosaic is completely based upon a number of rendering engines based on various features of Eclipse: browsers, property editors and diagram editors. The architecture has also been successfully used where XMF is deployed as a web-application and the rendering engine is a standard web-browser (using various combinations of HTML and the Google Web Toolkit).
A DSL for Interactive Applications
This section presents a simple interactive application expressed using the xDSL and then shows it running. The following section shows how the xDSL is implemented in XMF.Textual languages are developed in XMF by extending the basic language with new language features. XMF has a small basic language; the rest is developed by extension. Each new language feature starts with an '@' character; the feature may be used wherever any normal language construct is expected. In this way, the XMF engine can be developed into any special purpose DSL engine. The following fragment shows the start of an interactive application:
@Model Quiz
// The Quiz model describes an interactive application
// for a TV quiz. Viewers are presented with a sequence
// of questions and get a final score...
score : Integer;
// Screen definitions...
end
screen START()
vertical
text Welcome to the Quiz. Click the button to Start end
button Start
go Question1()
end
end
end
screen Question1()
vertical
text What is the capital of England? end
options Choice
option London;
option Paris;
option Madrid;
end
// Question1 continues...
// ... Question1 continues...
horizontal
button Next
// Next action continues...
end
button Quit
go Quit()
end
end
end
Actions may be conditional, the conditional expression may refer to choice variables, values of variables passed to screens and the current state of the model instance. Actions may also produce displays (without having to go to a new screen) which allows variables to be scoped locally within an action. In the following, the Next action has two local displays that are used to respond to the choice:
// ...Next action continues...
if Choice = "London"
then
display
text Well Done end
button Next
score := score + 1;
go Question2()
end
end
else
display
text Wrong! Answer is London. end
button Next
go Question2()
end
end
end
The following shows a partial execution of this application. Since there is no rendering engine attached, the XML is printed and the responses encoded by hand:
<Screen>
<Vertical>
<Text text=' Welcome to the Quiz. Click the button to Start '/>
<Button name='Start'/>
</Vertical>
</Screen>
Start <-- Event from rendering engine
<Screen>
<Vertical>
<Text text=' What is the capital of England? '/>
<Options name='Choice'>
<Option name='London'/>
<Option name='Paris'/>
<Option name='Madrid'/>
</Options>
<Horizontal>
<Button name='Next'/>
<Button name='Quit'/>
</Horizontal>
</Vertical>
</Screen>
Next Choice=London <-- Event from rendering engine
<Screen>
<Text text=' Well Done '/>
<Button name='Next'/>
</Screen>
Next <-- Event from rendering engine
Implementation
The implementation of the xDSL has two main components: a syntax model and a grammar that transforms text to instances of the syntax model, and a semantics model that defines an execution engine. The syntax model defines a modeling language that defines an application type; an instance of the type is defined by the semantics model. This is a typical way to define a language: models represent things that can be performed in some sense. Performing the models produces instances whose behaviour is expressed by the model. Another way to think about this is that we aim to produce libraries of reusable interactive applications (instances of the syntax model). A run-time occurrence of an application is described by the semantic model.XMF provides facilities for working with text including grammars, XML parsers and XML formatters. The models have been developed using the XMF development engine XMF-Mosaic and then run on the basic engine.
Syntax Definition
The syntax for the DSL has two parts: the abstract syntax and the concrete syntax. The abstract syntax consists of a collection of models that are described below. The concrete syntax is defined by a collection of grammars that are defined at the end of this section.Figure shows the top-level model for the interactive application language. A model consists of a collection of properties and a collection of screens. Each property is defined by the model in figure ; it has a name and a classifier. XMF has a meta-model that provides features such as Class and Object. A type is called a Classifier in XMF; Integer, String, Set(Object) are all XMF classifiers.
Screens are shown in figure . A screen has a collection of arguments. An action may cause the application to make a transition to a screen in which case the transition can supply argument values to be used when calculating the display for the screen. Figure shows the model for displays. Each element of the display model can produce XML output that is understood by the rendering engine. In addition, some of the display elements are associated with actions that will be performed when the appropriate event is received by the xDSL engine.
The display elements of an application model refer to an XOCL class called Exp. This is used wherever an executable fragment of code is required. It allows features of the displays to be computed dynamically. For example when a transition to a screen is made, the names of the buttons may depend on the values of the arguments that are passed to the screen. This is the reason why a button has an exp: it is used to calculate the label on the button in terms of the variables that are in scope at the time. The Exp class is a way of importing the models for XMF expressions into the displays model.
Menus are shown in figure . Each menu has a label that is computed (again depending on the context) and an action. Actions are defined in figure . An action is either a transition to a new screen (Go), a conditional action (If), an update to a variable currently in scope (Update) or a local display.
The developer of an interactive application does not directly create instances of the abstract syntax model. The idea is that they write in a text language that is parsed to synthesize instances of the syntax model.
XMF allows any class to be extended with a grammar. The class then defines a new syntax construct that is completely integrated into the base language of XMF. Any number of classes can be added in this way and new syntax classes can build on existing syntax classes. The following is a simple example of a class definition that implements a simple guarded expression:
@NotNull [e1].m(e2,e3,e4)
else e5
end
@Class NotNull extends Sugar
@Attribute exp : String end
@Attribute name : String end
@Attribute isMessage : Boolean end
@Attribute args : Seq(Performable) end
@Attribute error : Performable end
@Constructor(exp,name,error) end
@Constructor(exp,name,args,error)
self.isMessage := true
end
@Grammar extends OCL.grammar
NotNull ::=
'[' e = Exp ']' '.' n = Name NotNullTail^(e,n) 'end'.
NotNullTail(e,n) ::=
'(' as = NotNullArgs ')' x = NotNullElse { NotNull(e,n,as,x) }
| x = NotNullElse { NotNull(e,n,x) }.
NotNullArgs ::=
e = Exp es = (',' Exp)* { Seq{e|es} }
| { Seq{} }.
NotNullElse ::=
'else' Exp
| { [| null |] }.
end
@Operation desugar():Performable
[| let notNullValue = <exp>
in if notNullValue = null
then <error>
else <if isMessage
then Send([| notNullValue |],name,args)
else [| notNullValue.<name> |]
end>
end
end
|]
end
end
- The class extends Sugar which means that it is defining a new syntax construct by providing an operation called 'desugar' whose responsibility is to turn an instance of NotNull into program code.
- The grammar definition extends the OCL grammar and thereby imports all of the basic grammar-rule definitions. This provides the rule for Exp which is the top-level grammar-rule for all language constructs.
- Each grammar-rule consists of a name and a body. The rule may optionally have arguments. The body consists of terminals (in ' and'), builtins such as Name, rule-calls (possibly with arguments) and actions (inside { and }). The rule actions are any program expression, in most cases they use class-constructors to create an instance of a named class.
- The desugar operation uses lifting-quotes ([| and |]) to create an instance of syntax-classes. The opposite of lifting is dropping (< and >) used to calculate syntax by evaluating a program expression.
context Model
@Grammar extends Property.grammar, Screen.grammar
Model ::= n = Name ps = Property* ss = Screen* 'end' {
Model(n,ps,ss)
}.
end
context Property extends OCL::OCL.grammar
@Grammar extends OCL.grammar
Property ::= n = Name ':' e = SimpleExp ';' {
Property(n,Drop(e))
}.
end
context Screen
@Grammar extends Menu.grammar, Display.grammar
Screen ::=
'screen' n = Name '(' as = ScreenArgs ')'
ms = Menu*
ds = Display*
'end' { Screen(n,as,DisplayScreen(ms,ds)) }.
ScreenArgs ::=
a = Name as = (',' Name)* { Seq{a|as} }
| { Seq{} }.
end
Another interesting feature of the menu item name rule is the use of 'lift' to transform a data element (in this case a string n) into an expression whose evaluation produces the original data element:
context Menu
@Grammar extends OCL.grammar
Menu ::= 'menu' n = MenuItemName is = MenuItem* 'end' {
Menu(n,is)
}.
MenuItemName ::=
n = Name { Exp(n.lift()) }
| e = SimpleExp { Exp(e,e.FV(),null) }.
MenuItem ::=
Menu
| 'item' n = MenuItemName a = Action 'end' { MenuItem(n,a) }.
end
context Display
@Grammar extends Action.grammar, OCL.grammar
Display ::=
Text
| Button
| Options
| Horizontal
| Vertical.
Text ::= 'text' t = Char* 'end' {
Text(Exp(t.asString().lift()))
}.
Button ::=
'button' n = ComputedName
as = Action*
'end' { Button(n,as) }.
ComputedName ::=
n = Name { Exp(n.lift()) }
| e = SimpleExp { Exp(e,e.FV(),null) }.
Options ::=
'options' n = ComputedName
os = Option*
'end' { Options(n,os) }.
Option ::=
'option' n = Name ';' { n }.
Horizontal ::=
'horizontal'
ds = Display*
'end' { Horizontal(ds) }.
Vertical ::=
'vertical'
ds = Display*
'end' { Vertical(ds) }.
end
contxt Action
@Grammar extends OCL.grammar
Action ::=
UpdateProperty
| IfAction
| Go
| DisplayScreen.
DisplayScreen ::=
'display'
ms = Menu*
ds = Display*
'end' { DisplayScreen(ms,ds) }.
UpdateProperty ::=
n = Name ':=' e = SimpleExp ';' {
Update(n,Exp(e,e.FV(),null))
}.
Go ::= 'go' n = Name '(' as = GoArgs ')' { Go(n,as) }.
GoArgs ::=
e = GoArg es = (',' GoArg)* { Seq{e|es} }
| { Seq{} }.
GoArg ::= e = SimpleExp { Exp(e) }.
IfAction ::=
'if' e = SimpleExp
'then' d = Action
IfActionTail^(e,d).
IfActionTail(e,d1) ::=
'else' d2 = Action 'end' { If(Exp(e,e.FV(),null),d1,d2) }
| 'end' { If(Exp(e,e.FV(),null),d1,null) }.
end
Semantic Definition
The class Engine defines the execution engine. An engine controls an instance and maintains an id-table that maps event ids to handlers as shown below. The idea is that each time an event occurs, a handler from the id-table is used to produce a new XML display that is sent to the rendering engine. The XML data is calculated from the information in the model and the current state of the instance slots.
The rest of this section defines the engine execution semantics. The following section shows how the engine is connected to a rendering engine. The engine processes a screen-transition using the 'go' operation defined below. XMF supports many types of input and output channel. The 'go' operation shows an example of a string output channel used to capture and then return the XML output data as a string:
context Engine
@Operation go(screen:String,args:Seq(Element))
let sout = StringOutputChannel()
in instance.go(screen,args,self,sout);
sout.getString()
end
end
context Instance
@Operation go(screen:String,args:Seq(Element),engine:Engine,out:OutputChannel)
@NotNull [model.indexScreensByName(screen,null)].display(self,args,engine,out)
else self.error("No screen called " + screen)
end
end
context Screen
@Operation display(instance,args,engine,out)
display.display(instance,self.env(instance,args),engine,out)
end
context Screen
@Operation env(instance,values)
let env = args.zip(values)
in instance.slots()->iterate(slot env = env |
env.bind(slot.property().name(),slot.value()))
end
end
Each 'display' operation shows the use of an XMF language feature @XML ... end that is used to write XML output to a channel. The construct has the form:
@XML(out)
<TAG ATTS>
.. program code ...
</TAG>
end
context DisplayScreen
@Operation display(instance,env,engine,out)
@XML(out)
<Screen>
@For menu in menus do
menu.display(instance,env,engine,out)
end
@For display in displays do
display.display(instance,env,engine,out)
end
</Screen>
end
end
context Text
@Operation display(instance,env,engine,out)
@XML(out)
<"Text" text=exp.keyApply(env)/>
end
end
context Button
@Operation display(instance,env,engine,out)
let id = exp.keyApply(env)
in engine.registerActions(id,instance,env,actions);
@XML(out)
<Button name=id/>
end
end
end
context Horizontal
@Operation display(instance,env,engine,out)
@XML(out)
<Horizontal>
@For display in displays do
display.display(instance,env,engine,out)
end
</Horizontal>
end
end
context Options
@Operation display(instance,env,engine,out)
@XML(out)
<Options name=exp.keyApply(env)>
@For option in options do
@XML(out)
<Option name=option/>
end
end
</Options>
end
end
context Engine
@Operation registerActions(id,instance,env1,actions)
idTable.put(id,
@Operation(env2)
let value = null
in @For action in actions do
value := action.perform(instance,env2 + env1,self)
end;
value
end
end)
end
context If
@Operation perform(instance,env,engine)
if exp.keyApply(env + instance.env())
then thenAction.perform(instance,env,engine)
else @NotNull [elseAction].perform(instance,env,engine) end
end
end
context Update
@Operation perform(instance,env,engine)
@NotNull [instance.getSlot(name)].setValue(exp.keyApply(env + instance.env()))
else env.set(name,exp.keyApply(env + instance.env()))
end
end
context Go
@Operation perform(instance,env,engine)
engine.go(name,exps->collect(exp | exp.keyApply(env)))
end
context Display
@Operation perform(instance,env,engine)
let sout = StringOutputChannel()
in display.perform(instance,env,engine,sout);
sout.getString()
end
end
Handling Events
Events occur when the user interacts with the rendering engine, for example by pressing a button. When the event occurs, the current screen may contain any number of option groups. Each option group is named and offers a number of alternative values. The selected option may affect the behaviour of the engine in terms of variable updates and screen transitions. Therefore, the event sent from the rendering engine to the xDSL engine must encode the value of any option variables currently displayed.In addition there may be any number of ways an event can be raised: menu selection or button press. Each must be uniquely identified and the event must supply the identifier of the event that occurred.
An event is defined to have a format that starts with the event id and is followed by any number of option variable/value pairs:
<ID> <VAR>=<VALUE> ... <VAR>=<VALUE>
context Engine
@Grammar
Event ::= n = Name e = Binding* { Seq{n|e} }.
Binding ::= n = Name '=' v = Name { Seq{n|v} }.
end
context Engine
@Operation getDisplay(event:String)
let pair = Engine.grammar.parseString(event,"Event",Seq{}) then
id = pair->head;
env = pair->tail
in @TableGet handler = idTable[id] do
idTable.clear();
handler(env)
else self.error("No handler for " + name)
end
end
end
Simulation
Figure shows the architecture of an interactive application. The rendering engine is external to the design of an xDSL; the relationship between the two is defined by the XML schema for the display language and the format of event strings. However, it is useful to be able to simulate the rendering engine in order to test the xDSL engine. This can be done by setting up a simple test harness for a pair of data consumers and linking the xDSL engine with a rendering engine simulation that reads events strings in from the terminal. The following class implements a data producer-consumer pair:@Class Consumer
@Attribute filter : Operation end
@Attribute other : Consumer (!) end
@Constructor(filter) ! end
@Operation consume(data)
other.consume(filter(data))
end
end
@Operation mk_xDSL_filter(model:Model)
let engine = Engine(model.new())
in @Operation(event)
engine.getDisplay(event)
end
end
end
@Operation renderFilter(xml:String)
xml.asXML().pprint(stdout);
"".println();
stdin.skipWhiteSpace();
stdin.readLine().stripTrailingWhiteSpace()
end
@Operation simulate(model:Model)
let eConsumer = Consumer(mk_xDSL_filter(model));
dConsumer = Consumer(renderFilter)
in eConsumer.setOther(dConsumer);
dConsumer.setOther(eConsumer);
eConsumer.consume("START")
end
end
XML Representation for Applications
A requirement for interactive applications is to be able to dynamically update the content and to be able to transfer the content from remote locations in a standard format. The application describes in this paper is completely represented in data. This means that, although the application is executable, it can easily be serialized, sent over a communications link, and then uploaded onto the device that is running the xDSL engine.XMF provides support for encoding any data elements as XML. There is a basic markup provided for all XMF data; the mechanisms for which can easily be extended to provide bespoke XML encoding. Using the basic mechanisms, a model can be encoded as follows:
@WithOpenFile(fout -> "c:/model.xml")
let xout = XMLOutputChannel(fout,NameSpaceXMLFormatter())
in xout.writeValue(model)
end
end
The resulting output is produced in an XML file that is shown in figure . The XML markup shown in the figure is the default vanilla-flavour markup provided by XMF. It is possible to add adapters to an XML output channel that filter the data as it is processed and thereby use domain-specific markup. So instead of seeing Object and Slot as element tags, we might use Screen and Button.
The XML can be read back in using the following code:
@WithOpenFile(fin <- ModelFile)
let xin = XMLInputChannel(fin,NameSpaceXMLInflater())
in xin.parse()
end
end