Difference between revisions of "External controls api"

From MMS Wiki
Jump to: navigation, search
(Events:)
(Commands:)
 
(21 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Usage: ==
+
== Displaying an embed ==
 +
 
 +
For web auto-resize to work, the page should load JQuery and the following HTML tags:
 +
  <script src="<SERVER URL>/scripts/embedder.js"></script>   
 +
  <iframe src="<YOUR LTI LAUNCH>" onload="mms_resizer(this,null)"></iframe>
 +
 
 +
If the iframe is inside a scrollable container, a css selector for it needs to be passed as a third argument to mms_resizer as a string (ex: '#container' or '.ifc')
 +
If the embed needs to fit within a fixed size and auto-resize is not needed, embedder.js and mms_resizer are not needed.
 +
 
 +
For iOS and Android, just open the LTI launch inside a WebView.
 +
 
 +
== API Usage: ==
  
 
On iOS and Android webViews, there is direct access to window object. As such, everything works via "window.mms_embed_controls" object as outlined in the next two sections.
 
On iOS and Android webViews, there is direct access to window object. As such, everything works via "window.mms_embed_controls" object as outlined in the next two sections.
Line 6: Line 17:
  
 
Commands can be called as:
 
Commands can be called as:
   iframe.postMessage({ id: 'matchmysound-embed-controls-command',
+
   iframe.contentWindow.postMessage({ id: 'matchmysound-embed-controls-command',
 
                       fname: '<command name>', args: [<arguments>], uid: <uid string> },'*');
 
                       fname: '<command name>', args: [<arguments>], uid: <uid string> },'*');
  
Line 35: Line 46:
 
         hasRecording: // returns true when ‘matched’ is an option
 
         hasRecording: // returns true when ‘matched’ is an option
 
         setPlayTrack: // either ‘etalon’, ‘matched’, 'backing' or 'metronome'
 
         setPlayTrack: // either ‘etalon’, ‘matched’, 'backing' or 'metronome'
         startStopPlay: // Start/stop playback
+
         startStopPlay: // Start/stop playback. Takes an optional (bool) parameter where t=start and f=stop
 
    
 
    
 
         tunerModal: // open/close a modal with the tuner.  
 
         tunerModal: // open/close a modal with the tuner.  
Line 45: Line 56:
 
    
 
    
 
         getPartInfo: // Returns info object { name: <string>, instrument: <string>, can_record: <bool>,
 
         getPartInfo: // Returns info object { name: <string>, instrument: <string>, can_record: <bool>,
                     // default_bpm: <int>, has_backing: <bool>, has_full: <bool>,
+
                     // default_bpm: <int>, beats_per_bar: <int>, has_backing: <bool>, has_full: <bool>,
 
                     // multiple: <bool>, non_pitched: <bool> }
 
                     // multiple: <bool>, non_pitched: <bool> }
 
    
 
    
Line 66: Line 77:
 
         setLoop: // turn looping on/off. Only applies if a section is marked and shown.
 
         setLoop: // turn looping on/off. Only applies if a section is marked and shown.
 
          
 
          
 +
        hasPermissions, // returns the current state: null if unknown, true if have permissions and false if request failed. Takes optional (bool): if true, requests permissions.
 +
 
 
         canRecord, // Returns true if record button should be available and false otherwise
 
         canRecord, // Returns true if record button should be available and false otherwise
 
         setRecordBacking,  // Set the mode for recording:  'none'/'backing'/'metronome'
 
         setRecordBacking,  // Set the mode for recording:  'none'/'backing'/'metronome'
 
         setPageTurnMode: // Set "just page turn" mode value - true or false
 
         setPageTurnMode: // Set "just page turn" mode value - true or false
         startStopRecord: // Start/stop recording
+
         startStopRecord: // Start/stop recording. Takes an optional (bool) parameter where t=start and f=stop
 +
       
 +
        loadFileAsRecording: // opens a 'upload file' dialog and then processes the chosen file as if it was recorded
 
    
 
    
 
         setOptions: // takes two parameters: key and value. Choices for key are:
 
         setOptions: // takes two parameters: key and value. Choices for key are:
Line 76: Line 91:
 
                     //  "no_mouse_track_change": <bool> // disable mouse clicks changing to etalon/matched
 
                     //  "no_mouse_track_change": <bool> // disable mouse clicks changing to etalon/matched
 
                     //  "hide_alerts": <bool> // hide ui alerts so they could be displayed outside - Do not disable otherwise! (experimental)
 
                     //  "hide_alerts": <bool> // hide ui alerts so they could be displayed outside - Do not disable otherwise! (experimental)
                     //  "hide_cursor": <bool> // hide cursor (for follow me mode recording, for instance)
+
                     //  "hide_cursor": <bool> // hide cursor during recording
 +
                    //  "auto_rotate": <bool> // auto-rotate to scrolling mode if in landscape or if two systems do not fit the visible iframe area. (default: true)
 +
                    //  "countin_bars": <int> // the number of count-in bars to give before recording (<=0 is not allowed and defaults to 2 then)
 
    
 
    
 
         setShowFeedbackModal: // Set whether the modal for results (or "too_short"/"just_noise") is displayed. values: true/false
 
         setShowFeedbackModal: // Set whether the modal for results (or "too_short"/"just_noise") is displayed. values: true/false
Line 85: Line 102:
 
             /* Feedback object is guaranteed to have: {  
 
             /* Feedback object is guaranteed to have: {  
 
                 score: <total score>,
 
                 score: <total score>,
                 pitch_score: <pitch score>,
+
                 pitch_score: <pitch score>, timing_score: <timing score>, // The main two sub-scores that combine into the above 'score'
                 rhythm_score: <rhythm score>,
+
                 rhythm_score: <rhythm score>, tempo_score: <tempo score>, // These are aggregated together in the timing score.
 
                 avg_tempo: <tempo played at, % relative to original>
 
                 avg_tempo: <tempo played at, % relative to original>
 
                 duration: <duration of recording in seconds>,
 
                 duration: <duration of recording in seconds>,
Line 92: Line 109:
 
                 complete: <was it played to the end>,
 
                 complete: <was it played to the end>,
 
                 verdict: <textual feedback>, verdict_id: <text id of verdict for self-translation>,
 
                 verdict: <textual feedback>, verdict_id: <text id of verdict for self-translation>,
                 pass: <did it fit all the assignment requirements>
+
                 pass: <did it fit all the assignment requirements>,
 +
                attempt_id: <for reporting issues with feedback, in conjunction with submitDebug>
 
               } */
 
               } */
 +
        clearFeedback: // Clears the current student recording as well as feedback
 
   
 
   
 
         zoomCmd: // controls the zoom level of scroll note. takes as input a string command which is one of 'in'/'out'/'full'/'reset'
 
         zoomCmd: // controls the zoom level of scroll note. takes as input a string command which is one of 'in'/'out'/'full'/'reset'
Line 99: Line 118:
 
         canSubmitAudio: // check if a recording is available and enabled and not already submitted etc...
 
         canSubmitAudio: // check if a recording is available and enabled and not already submitted etc...
 
         submitAudio: // actually submit/save the audio
 
         submitAudio: // actually submit/save the audio
 +
       
 +
        submitDebug: // send the audio and the feedback to MMS for debugging. This is required for reporting issues with the feedback algorithm.
 
   
 
   
 
         // NB! WHAT FOLLOWS IS FOR TESTING ONLY! WILL NOT BE IN THE PRODUCTION RELEASE!  
 
         // NB! WHAT FOLLOWS IS FOR TESTING ONLY! WILL NOT BE IN THE PRODUCTION RELEASE!  
Line 107: Line 128:
  
 
You can also register a listener callback to certain events via an "on" function. Currently three events are available:  mms_embed_controls.on(‘<evt name>’, callbackFunction);
 
You can also register a listener callback to certain events via an "on" function. Currently three events are available:  mms_embed_controls.on(‘<evt name>’, callbackFunction);
 +
Events are dispatched only after the page is loaded so it is safe to initialize the listeners on window.onload
 
Callback function can sometimes have a parameter passed to it.
 
Callback function can sometimes have a parameter passed to it.
  
Line 128: Line 150:
 
* ‘mic_volume' - microphone volume for VU meter. Parameter is the value in range [0,1]
 
* ‘mic_volume' - microphone volume for VU meter. Parameter is the value in range [0,1]
 
* ‘alert' - alert information in parameter in json form, { 'type':<info/success/warning/error>,  'msg':<message>, 'msg_id':<id of message for translation> }
 
* ‘alert' - alert information in parameter in json form, { 'type':<info/success/warning/error>,  'msg':<message>, 'msg_id':<id of message for translation> }
 +
 +
* ‘permission_requested' - requesting recording permissions from user
 +
* ‘permission_received' - received recording permissions from user
 +
* ‘permission_not_received' - recording permissions not given or not available for other reasons (no microphone)
 +
 +
== Things to know ==
 +
 +
Which part is automatically loaded and which recording settings are set depend on what the user has done previously. If you want it to always open up a fixed part with fixed settings instead, you have to ensure that yourself!
 +
 +
== Standalone tuner: ==
 +
In case you need a stand-alone tuner, use the /tuner.html url.
 +
If you want it to autostart, /tuner.html#?autostart=true

Latest revision as of 06:04, 12 December 2018

Displaying an embed

For web auto-resize to work, the page should load JQuery and the following HTML tags:

 <script src="<SERVER URL>/scripts/embedder.js"></script>     
 <iframe src="<YOUR LTI LAUNCH>" onload="mms_resizer(this,null)"></iframe>

If the iframe is inside a scrollable container, a css selector for it needs to be passed as a third argument to mms_resizer as a string (ex: '#container' or '.ifc') If the embed needs to fit within a fixed size and auto-resize is not needed, embedder.js and mms_resizer are not needed.

For iOS and Android, just open the LTI launch inside a WebView.

API Usage:

On iOS and Android webViews, there is direct access to window object. As such, everything works via "window.mms_embed_controls" object as outlined in the next two sections.

For cross-origin uses in HTML5 with iframes, the message passing api (postMessage) needs to be used instead.

Commands can be called as:

 iframe.contentWindow.postMessage({ id: 'matchmysound-embed-controls-command',
                      fname: '<command name>', args: [<arguments>], uid: <uid string> },'*');

with the events being fired directly on the window object so they can be read via

 window.addEventListener('message', function(evt) {
    if (evt.data.id != 'matchmysound-embed-controls-message') return;
    if (evt.data.ename == '<event name>') { /* do stuff. Callback argument is evt.data.evalue */ } 
 });

If command returns a value, it is fired as a message similar to the events with the structure:

 {
   id: 'matchmysound-embed-controls-result',
   fname: <name of function called>,  args: <arguments called with>, uid: <uid echoed back>,
   result: <result that was returned>
 }

If the command does not exist, a message with id: 'matchmysound-embed-controls-result-error' is fired


Commands:

NB! All setters also function as getters when called without parameters, i.e. calling setXXX functions return current setting when called with ()

 window.mms_embed_controls = {
       toBeginning, // Scroll cursor to beginning. Optionally takes an argument ‘section’/’part’. 
                    // By default chooses which based on current cursor pos
 
       hasRecording: // returns true when ‘matched’ is an option
       setPlayTrack: // either ‘etalon’, ‘matched’, 'backing' or 'metronome'
       startStopPlay: // Start/stop playback. Takes an optional (bool) parameter where t=start and f=stop
 
       tunerModal: // open/close a modal with the tuner. 
                   // Takes an optional parameter (bool) and toggles if it is not given
       
       infoModal: // Display exercise information
 
       partModal: // Display part choice modal
 
       getPartInfo: // Returns info object { name: <string>, instrument: <string>, can_record: <bool>,
                    // default_bpm: <int>, beats_per_bar: <int>, has_backing: <bool>, has_full: <bool>,
                    // multiple: <bool>, non_pitched: <bool> }
 
       listParts: // Returns a list of part names
       setPart: // Sets the current part. Takes the name of the part as argument. NB! Needs time to load again
 
       setViewType: // Choices are 'page'/'scroll'/'wf'
       
       tempoModal: // Display tempo change modal
       setTempo: // Sets tempo. Values allowed are from 0.3 (30%) to 3 (300%)
       
       sectionMarked: // Return whether a section is selected
       markSection: // Start marking a section from cursor position
       resetSection: // Reset currently selected section
       setSection: // Takes two arguments: beg and end of section in seconds, or a dict with { beg: , end: }
                   // Returns {name: <string>, beg: 

Events:

You can also register a listener callback to certain events via an "on" function. Currently three events are available: mms_embed_controls.on(‘<evt name>’, callbackFunction); Events are dispatched only after the page is loaded so it is safe to initialize the listeners on window.onload Callback function can sometimes have a parameter passed to it.

  • ‘external_controls_ready' - commands are ready to be called and events will fire, although loading is still going on.
  • ‘partially_loaded' - exercise information is loaded up. Waiting only for visuals, audio and etalon. Safe to start calling commands (except play/record)
  • ‘fully_loaded' - all required assets loaded. Fired both initially and whenever part changes
  • 'initial_loading_progress' - callback is passed an integer value in the range [0-100] indicating the progress of loading initial metadata
  • 'loading_progress' - callback is passed an integer value in the range [0-100] indicating the progress of full loading (audio, all systems, recording etalon)
  • 'track_loaded' - callback is passed the name of the track loaded ('etalon'/'backing'/'metronome'/'full')
  • 'etalon_loaded' - the ready event for recording button
  • ‘view_type_changed' - fired whenever view_type is either first set or changed
  • 'changed_part' - user has chosen a different part in a multi-part exercise
  • 'feedback_available' - feedback is avaliable, i.e. an attempt was just finished. Fired whenever recording stops.
  • ‘play_start' - audio playback started, either by pressing <space> or from ui
  • ‘play_end' - audio playback stopped, either automatically or from ui
  • ‘changed_active_track' - switched between ‘etalon’ and ‘matched’. Query setPlayTrack() to see current
  • ‘modals_showing' - a new modal open. Parameter true if it is a loading modal.
  • ‘modals_cleared' - all modals closed
  • ‘mic_volume' - microphone volume for VU meter. Parameter is the value in range [0,1]
  • ‘alert' - alert information in parameter in json form, { 'type':<info/success/warning/error>, 'msg':<message>, 'msg_id':<id of message for translation> }
  • ‘permission_requested' - requesting recording permissions from user
  • ‘permission_received' - received recording permissions from user
  • ‘permission_not_received' - recording permissions not given or not available for other reasons (no microphone)

Things to know

Which part is automatically loaded and which recording settings are set depend on what the user has done previously. If you want it to always open up a fixed part with fixed settings instead, you have to ensure that yourself!

Standalone tuner:

In case you need a stand-alone tuner, use the /tuner.html url. If you want it to autostart, /tuner.html#?autostart=true