Gesture Control for programmable displays with touchscreen

(caution: only available for certain terminals with touchscreen, see feature matrix)

Overview

See also:

Introduction

Since April 2012, some of MKT's user programmable terminals (which include the MKT-View family) have neither a real keyboard, nor a rotary pushbutton.
To operate these devices, for example enter the system menu, such devices may be controlled via simple touchscreen gestures. Shown below: Example for the gesture to open the system control window via gesture ("rectangular U"),
here with 'visible gestures' (= dotted line showing the trace) activated in the System menu under 'System Setup'..'Display'..'Vis. Gesture':

gesture with visible trace
Gesture 'U' on a 7 inch display, with visible trace (dotted lines)

Any gesture which the operator 'paints' (visibly or invisibly) on the touchscreen is scaled into a grid with nine fields, which are numbered as shown below:

9-field gesture area
Nine-field touchscreen gesture area

gesture to open the shutdown window / system menu
Gesture 'U', opens the shutdown-screen

A gesture begins when the finger (or touch pen) is pressed down on the display. It ends when the finger (or pen) is released. This is when the actual 'decoding' of the gesture takes place. The screen area (pixel coordinates) of the gesture area is not important for the detection.
Requirements for a successful gesture recognition are:

  • the gesture area should extend over at least 25 % of the touchscreen area
  • the gesture must be drawn in a single passage (no 'pen up' in between)
  • the gesture must be finished within five seconds (no 'very slow' gestures)
  • the gesture must consist only of vertical and horizontal movements (not diagonal)
  • the 'path' of a gesture is limited to 8 fields, limited by 4 bit per field in a 32-bit 'gesture code'.

Hard-coded gestures

The following gestures are reserved for the system. They have priority over user-defined gestures, and thus cannot be used to control your own display application:

gesture to open the virtual keyboard
Gesture 'ENTER', code 0x36987 (hex), used to open the virtual keyboard



gesture to open the shutdown window / system menu
Gesture 'U', code 0x1478963, used to open the shutdown-screen (or system menu)

3. Gestures available for the User Application

Simple gestures like 'wipe'-movements can be used to control the user application by means of the script language, or the 'programmable events' of individual display pages.
The hexadecimal codes shown below can be used in your application (see example further below).
Each code is a 4-bit-wise combination of the fields 'touched' by a gesture, beginning with the first field (start of the movement) in the leftmost (most significant) hex digit, and ending with the final field (end of the movement; "pen up") in the four least significant bits. The fields are numbered from 1 to 9 (shown in gray colour in the table below).

Example: The gesture "wipe from left to right" runs through the fields 1, 2, 3 (in this order), thus the hexadecimal code of this gesture is 0x123. The prefix '0x' indicates a hexadecimal number in almost every programming language, except for a few exceptions (Pascal).


Gesture 'Left to Right'
'Wipe from left to right',
Code 0x123 (hex)

Gesture 'Right to Left'
'Wipe from right to left',
Code 0x321 (hex)

Gesture 'Top to Bottom'
'Wipe from top to bottom',
Code 0x147 (hex)

Gesture 'Bottom to Top'
'Wipe from bottom to top',
Code 0x741 (hex)

To use these (and other) gestures in your own application, we recommend the select..case-construct in the script language. Implement an OnGesture handler in your script, and forget about the old 'programmable page events'. Here is a simple script fragment, which recognizes some of the simple 'wipe' gestures.


func OnGesture( int gestureCode, int gestureSize )
    // Called by the system after the operator "painted" a gesture on the touchscreen.
    // Parameter: gestureCode = gesture pattern, see manual (gestures_01.htm) .
    //            gestureSize = approximate size of the gesture painting area,
    //                          measured in "Percent of the screen width" .
    select gestureCode
       case 0x123 : // wiped from left to right
         display.goto_page( display.page_index - 1 ); // go to the PREVIOUS page
       case 0x321 : // wiped from right to left
         display.goto_page( display.page_index + 1 ); // go to the FOLLOWING page
       else       : // all other gestures are NOT handled here..
         return 0;  // ..so let someone else handle this event
    endselect;
    return 1;       // event has been handled here - don't handle it elsewhere
endfunc;

Last modifications:
  • 2012-04-02 : First implementation of touchscreen gestures in MKT's "UPT 800" (HMI with 7" touchscreen, w/o keyboard and rotary encoder)