Difference between revisions of "External controls api"

From MMS Wiki
Jump to: navigation, search
(Commands:)
(Commands:)
 
(36 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Usage: ==
+
== Displaying an embed ==
 +
 
 +
For web auto-resize to work, the page should load JQuery and the following HTML tags:
 +
  <script src="<SERVER URL>/scripts/embedder.js"></script>   
 +
  <iframe src="<YOUR LTI LAUNCH>" onload="mms_resizer(this,null)"></iframe>
 +
 
 +
If the iframe is inside a scrollable container, a css selector for it needs to be passed as a third argument to mms_resizer as a string (ex: '#container' or '.ifc')
 +
If the embed needs to fit within a fixed size and auto-resize is not needed, embedder.js and mms_resizer are not needed.
 +
 
 +
For iOS and Android, just open the LTI launch inside a WebView.
 +
 
 +
== API Usage: ==
  
 
On iOS and Android webViews, there is direct access to window object. As such, everything works via "window.mms_embed_controls" object as outlined in the next two sections.
 
On iOS and Android webViews, there is direct access to window object. As such, everything works via "window.mms_embed_controls" object as outlined in the next two sections.
Line 6: Line 17:
  
 
Commands can be called as:
 
Commands can be called as:
   iframe.postMessage({ id: 'matchmysound-embed-controls-command',
+
   iframe.contentWindow.postMessage({ id: 'matchmysound-embed-controls-command',
 
                       fname: '<command name>', args: [<arguments>], uid: <uid string> },'*');
 
                       fname: '<command name>', args: [<arguments>], uid: <uid string> },'*');
  
Line 35: Line 46:
 
         hasRecording: // returns true when ‘matched’ is an option
 
         hasRecording: // returns true when ‘matched’ is an option
 
         setPlayTrack: // either ‘etalon’, ‘matched’, 'backing' or 'metronome'
 
         setPlayTrack: // either ‘etalon’, ‘matched’, 'backing' or 'metronome'
         startStopPlay: // Start/stop playback
+
         startStopPlay: // Start/stop playback. Takes an optional (bool) parameter where t=start and f=stop
 
    
 
    
 
         tunerModal: // open/close a modal with the tuner.  
 
         tunerModal: // open/close a modal with the tuner.  
Line 44: Line 55:
 
         partModal: // Display part choice modal
 
         partModal: // Display part choice modal
 
    
 
    
         getPartInfo: // Returns info object { name: <string>, can_record: <bool>,
+
         getPartInfo: // Returns info object { name: <string>, instrument: <string>, can_record: <bool>,
                     // default_bpm: <int>, has_backing: <bool>, has_full: <bool>,
+
                     // default_bpm: <int>, beats_per_bar: <int>, has_backing: <bool>, has_full: <bool>,
 
                     // multiple: <bool>, non_pitched: <bool> }
 
                     // multiple: <bool>, non_pitched: <bool> }
 
    
 
    
Line 59: Line 70:
 
         markSection: // Start marking a section from cursor position
 
         markSection: // Start marking a section from cursor position
 
         resetSection: // Reset currently selected section
 
         resetSection: // Reset currently selected section
 +
        setSection: // Takes two arguments: beg and end of section in seconds, or a dict with { beg: , end: }
 +
                    // Returns {name: <string>, beg: <time in sec>, end: <time in sec>}
 +
                    // name is null unless beg and end match a named section exactly
 +
        listSections: // Returns list of sections in the above return format, including the "Full" section
 
          
 
          
 
         setLoop: // turn looping on/off. Only applies if a section is marked and shown.
 
         setLoop: // turn looping on/off. Only applies if a section is marked and shown.
 
          
 
          
 +
        hasPermissions, // returns the current state: null if unknown, true if have permissions and false if request failed. Takes optional (bool): if true, requests permissions.
 +
 
 
         canRecord, // Returns true if record button should be available and false otherwise
 
         canRecord, // Returns true if record button should be available and false otherwise
 
         setRecordBacking,  // Set the mode for recording:  'none'/'backing'/'metronome'
 
         setRecordBacking,  // Set the mode for recording:  'none'/'backing'/'metronome'
 
         setPageTurnMode: // Set "just page turn" mode value - true or false
 
         setPageTurnMode: // Set "just page turn" mode value - true or false
         startStopRecord: // Start/stop recording
+
         startStopRecord: // Start/stop recording. Takes an optional (bool) parameter where t=start and f=stop
 
    
 
    
 
         setOptions: // takes two parameters: key and value. Choices for key are:
 
         setOptions: // takes two parameters: key and value. Choices for key are:
Line 71: Line 88:
 
                     //  "ignore_keyboard": <bool> // disable reacting to keyboard events (allowing them to be caught outside)
 
                     //  "ignore_keyboard": <bool> // disable reacting to keyboard events (allowing them to be caught outside)
 
                     //  "no_mouse_track_change": <bool> // disable mouse clicks changing to etalon/matched
 
                     //  "no_mouse_track_change": <bool> // disable mouse clicks changing to etalon/matched
 +
                    //  "hide_alerts": <bool> // hide ui alerts so they could be displayed outside - Do not disable otherwise! (experimental)
 +
                    //  "hide_cursor": <bool> // hide cursor during recording
 +
                    //  "auto_rotate": <bool> // auto-rotate to scrolling mode if in landscape or if two systems do not fit the visible iframe area. (default: true)
 +
                    //  "countin_bars": <int> // the number of count-in bars to give before recording (<=0 is not allowed and defaults to 2 then)
 
    
 
    
 
         setShowFeedbackModal: // Set whether the modal for results (or "too_short"/"just_noise") is displayed. values: true/false
 
         setShowFeedbackModal: // Set whether the modal for results (or "too_short"/"just_noise") is displayed. values: true/false
Line 81: Line 102:
 
                 pitch_score: <pitch score>,
 
                 pitch_score: <pitch score>,
 
                 rhythm_score: <rhythm score>,
 
                 rhythm_score: <rhythm score>,
 +
                avg_tempo: <tempo played at, % relative to original>
 +
                duration: <duration of recording in seconds>,
 
                 percent_played: <percentage of the piece played>,
 
                 percent_played: <percentage of the piece played>,
                 duration: <duration of recording in seconds>,
+
                 complete: <was it played to the end>,
 +
                verdict: <textual feedback>, verdict_id: <text id of verdict for self-translation>,
 +
                pass: <did it fit all the assignment requirements>,
 +
                attempt_id: <for reporting issues with feedback, in conjunction with submitDebug>
 
               } */
 
               } */
 +
        clearFeedback: // Clears the current student recording as well as feedback
 
   
 
   
 
         zoomCmd: // controls the zoom level of scroll note. takes as input a string command which is one of 'in'/'out'/'full'/'reset'
 
         zoomCmd: // controls the zoom level of scroll note. takes as input a string command which is one of 'in'/'out'/'full'/'reset'
 +
       
 +
        canSubmitAudio: // check if a recording is available and enabled and not already submitted etc...
 +
        submitAudio: // actually submit/save the audio
 +
       
 +
        submitDebug: // send the audio and the feedback to MMS for debugging. This is required for reporting issues with the feedback algorithm.
 
   
 
   
 
         // NB! WHAT FOLLOWS IS FOR TESTING ONLY! WILL NOT BE IN THE PRODUCTION RELEASE!  
 
         // NB! WHAT FOLLOWS IS FOR TESTING ONLY! WILL NOT BE IN THE PRODUCTION RELEASE!  
         setColors: takes as input a dictionary {background:<color>, cursor:<color>, section:<color>} where <color> is a valid CSS color (like '#FF0000' or 'hsb(0,0,0)')  
+
         setColors: takes as input a dictionary {background:<color>, cursor:<color>, section:<color>, overlay:<color>} where <color> is a valid CSS color (like '#FF0000' or 'hsb(0,0,0)') ,
 
       };
 
       };
  
Line 94: Line 126:
  
 
You can also register a listener callback to certain events via an "on" function. Currently three events are available:  mms_embed_controls.on(‘<evt name>’, callbackFunction);
 
You can also register a listener callback to certain events via an "on" function. Currently three events are available:  mms_embed_controls.on(‘<evt name>’, callbackFunction);
 +
Events are dispatched only after the page is loaded so it is safe to initialize the listeners on window.onload
 +
Callback function can sometimes have a parameter passed to it.
  
 
* ‘external_controls_ready' - commands are ready to be called and events will fire, although loading is still going on.
 
* ‘external_controls_ready' - commands are ready to be called and events will fire, although loading is still going on.
Line 100: Line 134:
 
* 'initial_loading_progress' - callback is passed an integer value in the range [0-100] indicating the progress of loading initial metadata
 
* 'initial_loading_progress' - callback is passed an integer value in the range [0-100] indicating the progress of loading initial metadata
 
* 'loading_progress' - callback is passed an integer value in the range [0-100] indicating the progress of full loading (audio, all systems, recording etalon)
 
* 'loading_progress' - callback is passed an integer value in the range [0-100] indicating the progress of full loading (audio, all systems, recording etalon)
 +
* 'track_loaded' - callback is passed the name of the track loaded ('etalon'/'backing'/'metronome'/'full')
 +
* 'etalon_loaded' - the ready event for recording button
  
 
* ‘view_type_changed' - fired whenever view_type is either first set or changed
 
* ‘view_type_changed' - fired whenever view_type is either first set or changed
Line 108: Line 144:
 
* ‘play_end' - audio playback stopped, either automatically or from ui
 
* ‘play_end' - audio playback stopped, either automatically or from ui
 
* ‘changed_active_track' - switched between ‘etalon’ and ‘matched’. Query setPlayTrack() to see current
 
* ‘changed_active_track' - switched between ‘etalon’ and ‘matched’. Query setPlayTrack() to see current
* ‘modals_showing' - at least one modal open
+
* ‘modals_showing' - a new modal open. Parameter true if it is a loading modal.
 
* ‘modals_cleared' - all modals closed
 
* ‘modals_cleared' - all modals closed
* ‘mic_volume' - microphone volume for VU meter. In range [0,1]
+
* ‘mic_volume' - microphone volume for VU meter. Parameter is the value in range [0,1]
 +
* ‘alert' - alert information in parameter in json form, { 'type':<info/success/warning/error>,  'msg':<message>, 'msg_id':<id of message for translation> }
 +
 
 +
* ‘permission_requested' - requesting recording permissions from user
 +
* ‘permission_received' - received recording permissions from user
 +
* ‘permission_not_received' - recording permissions not given or not available for other reasons (no microphone)
 +
 
 +
== Things to know ==
 +
 
 +
Which part is automatically loaded and which recording settings are set depend on what the user has done previously. If you want it to always open up a fixed part with fixed settings instead, you have to ensure that yourself!
 +
 
 +
== Standalone tuner: ==
 +
In case you need a stand-alone tuner, use the /tuner.html url.
 +
If you want it to autostart, /tuner.html#?autostart=true

Latest revision as of 23:29, 26 September 2018

Displaying an embed

For web auto-resize to work, the page should load JQuery and the following HTML tags:

 <script src="<SERVER URL>/scripts/embedder.js"></script>     
 <iframe src="<YOUR LTI LAUNCH>" onload="mms_resizer(this,null)"></iframe>

If the iframe is inside a scrollable container, a css selector for it needs to be passed as a third argument to mms_resizer as a string (ex: '#container' or '.ifc') If the embed needs to fit within a fixed size and auto-resize is not needed, embedder.js and mms_resizer are not needed.

For iOS and Android, just open the LTI launch inside a WebView.

API Usage:

On iOS and Android webViews, there is direct access to window object. As such, everything works via "window.mms_embed_controls" object as outlined in the next two sections.

For cross-origin uses in HTML5 with iframes, the message passing api (postMessage) needs to be used instead.

Commands can be called as:

 iframe.contentWindow.postMessage({ id: 'matchmysound-embed-controls-command',
                      fname: '<command name>', args: [<arguments>], uid: <uid string> },'*');

with the events being fired directly on the window object so they can be read via

 window.addEventListener('message', function(evt) {
    if (evt.data.id != 'matchmysound-embed-controls-message') return;
    if (evt.data.ename == '<event name>') { /* do stuff. Callback argument is evt.data.evalue */ } 
 });

If command returns a value, it is fired as a message similar to the events with the structure:

 {
   id: 'matchmysound-embed-controls-result',
   fname: <name of function called>,  args: <arguments called with>, uid: <uid echoed back>,
   result: <result that was returned>
 }

If the command does not exist, a message with id: 'matchmysound-embed-controls-result-error' is fired


Commands:

NB! All setters also function as getters when called without parameters, i.e. calling setXXX functions return current setting when called with ()

 window.mms_embed_controls = {
       toBeginning, // Scroll cursor to beginning. Optionally takes an argument ‘section’/’part’. 
                    // By default chooses which based on current cursor pos
 
       hasRecording: // returns true when ‘matched’ is an option
       setPlayTrack: // either ‘etalon’, ‘matched’, 'backing' or 'metronome'
       startStopPlay: // Start/stop playback. Takes an optional (bool) parameter where t=start and f=stop
 
       tunerModal: // open/close a modal with the tuner. 
                   // Takes an optional parameter (bool) and toggles if it is not given
       
       infoModal: // Display exercise information
 
       partModal: // Display part choice modal
 
       getPartInfo: // Returns info object { name: <string>, instrument: <string>, can_record: <bool>,
                    // default_bpm: <int>, beats_per_bar: <int>, has_backing: <bool>, has_full: <bool>,
                    // multiple: <bool>, non_pitched: <bool> }
 
       listParts: // Returns a list of part names
       setPart: // Sets the current part. Takes the name of the part as argument. NB! Needs time to load again
 
       setViewType: // Choices are 'page'/'scroll'/'wf'
       
       tempoModal: // Display tempo change modal
       setTempo: // Sets tempo. Values allowed are from 0.3 (30%) to 3 (300%)
       
       sectionMarked: // Return whether a section is selected
       markSection: // Start marking a section from cursor position
       resetSection: // Reset currently selected section
       setSection: // Takes two arguments: beg and end of section in seconds, or a dict with { beg: , end: }
                   // Returns {name: <string>, beg: 

Events:

You can also register a listener callback to certain events via an "on" function. Currently three events are available: mms_embed_controls.on(‘<evt name>’, callbackFunction); Events are dispatched only after the page is loaded so it is safe to initialize the listeners on window.onload Callback function can sometimes have a parameter passed to it.

  • ‘external_controls_ready' - commands are ready to be called and events will fire, although loading is still going on.
  • ‘partially_loaded' - exercise information is loaded up. Waiting only for visuals, audio and etalon. Safe to start calling commands (except play/record)
  • ‘fully_loaded' - all required assets loaded. Fired both initially and whenever part changes
  • 'initial_loading_progress' - callback is passed an integer value in the range [0-100] indicating the progress of loading initial metadata
  • 'loading_progress' - callback is passed an integer value in the range [0-100] indicating the progress of full loading (audio, all systems, recording etalon)
  • 'track_loaded' - callback is passed the name of the track loaded ('etalon'/'backing'/'metronome'/'full')
  • 'etalon_loaded' - the ready event for recording button
  • ‘view_type_changed' - fired whenever view_type is either first set or changed
  • 'changed_part' - user has chosen a different part in a multi-part exercise
  • 'feedback_available' - feedback is avaliable, i.e. an attempt was just finished. Fired whenever recording stops.
  • ‘play_start' - audio playback started, either by pressing <space> or from ui
  • ‘play_end' - audio playback stopped, either automatically or from ui
  • ‘changed_active_track' - switched between ‘etalon’ and ‘matched’. Query setPlayTrack() to see current
  • ‘modals_showing' - a new modal open. Parameter true if it is a loading modal.
  • ‘modals_cleared' - all modals closed
  • ‘mic_volume' - microphone volume for VU meter. Parameter is the value in range [0,1]
  • ‘alert' - alert information in parameter in json form, { 'type':<info/success/warning/error>, 'msg':<message>, 'msg_id':<id of message for translation> }
  • ‘permission_requested' - requesting recording permissions from user
  • ‘permission_received' - received recording permissions from user
  • ‘permission_not_received' - recording permissions not given or not available for other reasons (no microphone)

Things to know

Which part is automatically loaded and which recording settings are set depend on what the user has done previously. If you want it to always open up a fixed part with fixed settings instead, you have to ensure that yourself!

Standalone tuner:

In case you need a stand-alone tuner, use the /tuner.html url. If you want it to autostart, /tuner.html#?autostart=true