Announcement

Collapse
No announcement yet.

Enable voice control wherever a textbox/OSK exists? HOW??

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Enable voice control wherever a textbox/OSK exists? HOW??

    Does anyone know how I can enable Voice Control whenever the OSK/text Cursor appears?

    I dont want to control RR with it, only enter text and have it auto trigger whever there is an opportunity to type.

    EG:
    Enter address in Satnav
    Enter track title in music search
    Fill in Album/Artist/Genre in CdRip plugin
    Enter a phone number in Phoco/Freefone/RRmobile

    Is there an elegant/correct way to have this occur? something like trapping every call to OSK, or detecting wherever there is a text cursor/input field on the screen?
    Any other ideas?

    Thanks
    Ice

  • #2
    Only by a means of a plugin you'd be able to check if RR is in either NUMOSK.SKIN or OSK.SKIN and have the voice control activated... I'd suppose you could also have a timer in NUMOSK.SKIN and OSK.SKIN which executes a command to activate the voice, and then have another command in the ESC/ENTER buttons of the OSK so it disables the voice control.
    Ride Runner RR's Myspace

    "Being happy is not about having what you want, it's about wanting what you have."
    "The best things in life are always free - but that doesn't mean money can't buy you good things."

    Comment


    • #3
      Is there any way to detect if the pointer has become a text cursor ?
      and doing it that way?

      Comment


      • #4
        try this. simple very simple script.

        Code:
        #Include <Misc.au3>
        AutoItSetOption("WinTitleMatchMode", 3)
        $test = "test"
        If WinExists($test) Then Exit 
        AutoItWinSetTitle($test)
        
        $rr=ObjCreate("RoadRunner.sdk")
        if not IsObj($rr) then Exit
        $oMyError = ObjEvent("AutoIt.Error","MyErrFunc")
        
        HotKeySet("!{esc}", "_exit") ;for testing only, exit ALT+ESC disable when no need for it.
        
        While 1
        	If  $oMyError.windescription then 
        		if Not WinExists("RoadRunner") then ExitLoop ;exit if RR is not Running.
        	EndIf
        		
            if MouseGetCursor() = 5 And _IsPressed("01") then
              MsgBox(4096, "", " text mode")
               $rr.execute("YOUR_COMMAND") ;place here your command
        	EndIf
        Sleep (3000)                ;push and hold 3 sec for executing.
        WEnd
        ;==============================================================================================
        ;AI's error reporting Object
        ;==============================================================================================
        
        Func MyErrFunc()
            Local $err = $oMyError.number
            If $err = 0 Then $err = -1
            SetError($err) 
        Endfunc
        
        Func _exit()
        Exit
        EndFunc
        BASSRR ==> [Audio Player/MusicDB/Cd Player/Ripper/Rate/Internet Radio/Audio Capture/Youtube Video]
        RRTube ==> [View/Download Youtube Videos]
        WifiMan ==> [Wifi Manager]
        RRec ==> [Audio Capture]

        If You Want To Be Given Everything, Give Everything Up ...
        Bird York -- In The Deep --

        Comment


        • #5
          Perfect thread.....Subscribed to thread

          El Camino, do you have the same code for vb.net

          I think i can create the roadrunner.sdk object like this
          Code:
          Dim rr As Object = CreateObject("RoadRunner.sdk")
          hmm, strange but neat at the same time. If i do that roadrunner starts. so what..., i would be compiling a new program that would include RR? Or a new program that would include my features but call/open RR?

          Okay tested a little more. So That will give me access to send to RR? And if my program is run without rr running, then rr will start. Otherwise RoadRunner can LOAD my program while entering a skin that would need my programs enteraction. Then you can have the skin close my program on exit.....hmmm.

          Iceman_jkh, I would love to see what you end up with. I have a program that i'm secretly working on that needs the same kind of control of RR.
          Um, I guess this is where you put something witty.WITTY

          My Web site, in the design stage. http://home.comcast.net/~cstrachn

          Modified RRSkinEditor http://www.mp3car.com/vbulletin/showthread.php?t=65723

          Comment


          • #6
            Hope the forum lets me double post
            Anyways after a couple seconds i can send searchs to the search box like this.
            Created a new form with button1 and textbox1

            Code:
            Public Class Form1
            
                Dim rr As Object = CreateObject("RoadRunner.sdk")
            
                Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
                End Sub
            
                Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
                    Dim i As Integer
                    Dim ch As String ' must be string to send :)
                    For i = 0 To TextBox1.Text.Length - 1
                        ch = TextBox1.Text.Chars(i)
                        rr.execute(ch)
                    Next i
                    rr.execute("Enter")
                End Sub
            End Class
            Simple VB.net code
            Um, I guess this is where you put something witty.WITTY

            My Web site, in the design stage. http://home.comcast.net/~cstrachn

            Modified RRSkinEditor http://www.mp3car.com/vbulletin/showthread.php?t=65723

            Comment


            • #7
              Thanks El Camino.

              EDIT: I got it to compile - turns out it must be saved as a .au3 file before it allows compilation (DOH!)

              Basically, if the cursor becomes a text cursor, I want:
              a) Audio to mute/pause
              b) Mic to turn on
              c) Accept commands/phonetic alphabet...until;
              d) ENTER/command word is said (which is the equivalent of pressing ENTER)
              e) turn mic off - as ENTER should change the cursor back to a pointer (not text cursor).
              f) Audio resumes/unmutes

              Profit> would also love to see what you are working on (in secret :P)

              I'm hoping this little 'plugin' will make the voice control actually viable and useful!

              Cheers
              Ice

              Comment


              • #8
                Question ICE

                Originally posted by Iceman_jkh View Post
                Thanks El Camino.

                Can you step me through that script a little. I ve only written 1 very VERY basic script before. I couldnt even see how to get this to compile.

                EDIT: I got it to compile - turns out it must be saved as a .au3 file before it allows compilation (DOH!)

                Basically, if the cursor becomes a text cursor, I want:
                a) Audio to mute/pause
                b) Mic to turn on
                c) Accept commands/phonetic alphabet...until;
                d) ENTER/command word is said (which is the equivalent of pressing ENTER)
                e) turn mic off - as ENTER should change the cursor back to a pointer (not text cursor).
                f) Audio resumes/unmutes

                Profit> would also love to see what you are working on (in secret :P)

                I'm hoping this little 'plugin' will make the voice control actually viable and useful!

                Cheers
                Ice
                Ice - Won't you still have to issue the "letters" command or "numbers" command since they are in the commands.cfg or could you load a different config file for just the osk ? Also, doesn't the voice module stop listening after a successful command is issued and you have to prompt again ? I know I'm asking more questions than I'm helping but wanted to think it all the way through.

                I use the voice control in mine all the time but I have a steering wheel control mapped to enable the voice prompt and then mute the mic after a set amount of time. Works out pretty well - I don't normally issue just letter and number commands because of what I asked you above.

                Comment


                • #9
                  OK, now that I can run it , Ive managed to understand whats going on

                  Heres what ive put together.
                  Purely a concept at the moment. Credit goes to El Camino for the main code!

                  Code:
                  #Include <Misc.au3>
                  AutoItSetOption("WinTitleMatchMode", 3)
                  $test = "test"
                  $SpeechOn = -1
                  
                  If WinExists($test) Then Exit 
                  AutoItWinSetTitle($test)
                  
                  $rr=ObjCreate("RoadRunner.sdk")
                  if not IsObj($rr) then Exit
                  $oMyError = ObjEvent("AutoIt.Error","MyErrFunc")
                  
                  HotKeySet("!{esc}", "_exit") ;for testing only, exit ALT+ESC disable when no need for it.
                  
                  While 1
                  	If  $oMyError.windescription then 
                  		if Not WinExists("RoadRunner") then ExitLoop ;exit if RR is not Running.
                  	EndIf
                  	
                  	If MouseGetCursor() = 5 And _IsPressed("01") then
                  		;Place code here.. to tell Voice control to listen
                  	    ;$rr.execute("My_Commands")
                  		$rr.execute("next") ;Just a test - so you can see it switch to the next track - delete this line in real version!
                  	   $SpeechOn= 1 ; Tells the app that it is about to receive speech commands
                  	EndIf
                  	
                  Sleep (800)                ;push and hold for 0.8 seconds to engage Voice control
                  	
                  	While MouseGetCursor() = 5 And $SpeechOn = 1
                  		; While we are waiting for the speech input, we make sure that the cursor stays as a TEXT CURSOR and is in speech mode
                  		; Just sit here and do nothing. This means the app will wait till the 
                  		; ..user finishes talking and issues their "ENTER" command
                  		; Might have to add a trap for AND Not(_IsPressed("0D")) to the end of this While loop
                  	WEnd
                  	
                  	If Not(MouseGetCursor() = 5) And $SpeechOn = 1 Then ;As soon as the user presses ENTER or gives the ENTER command
                  	
                  		$SpeechOn= 0 ; Turn off voice commanding - ignore incoming commands/turn mic off
                  		$rr.execute("prev") ;Just a test - so you can see it switch to the prev track - delete this line in real version!
                  		;$rr.execute("My_Commands") any other commands you wish to run
                  	EndIf
                  	
                  			
                  WEnd
                  ;==============================================================================================
                  ;AI's error reporting Object
                  ;==============================================================================================
                  
                  Func MyErrFunc()
                      Local $err = $oMyError.number
                      If $err = 0 Then $err = -1
                      SetError($err) 
                  Endfunc
                  
                  Func _exit()
                  Exit
                  EndFunc
                  Compile it to an autoit script.
                  Hold left click for 0.8 seconds on a text box - should skip to next track.
                  Pressing enter, moving cursor off text box,etc should skip to prev track.

                  Basically all this does is prove that you can recognise such events and then trigger commands as a result
                  Next step is getting it to control/interface to the voice commands aspect of RR.

                  Ice

                  Comment


                  • #10
                    Originally posted by montgomeryt View Post
                    Ice - Won't you still have to issue the "letters" command or "numbers" command since they are in the commands.cfg or could you load a different config file for just the osk ? Also, doesn't the voice module stop listening after a successful command is issued and you have to prompt again ? I know I'm asking more questions than I'm helping but wanted to think it all the way through.

                    I use the voice control in mine all the time but I have a steering wheel control mapped to enable the voice prompt and then mute the mic after a set amount of time. Works out pretty well - I don't normally issue just letter and number commands because of what I asked you above.
                    Im not that advanced at the voice control prog as you are mate.
                    But reading the .xml I can see what you are asking.
                    Hopefully we can create a smaller version of the .xml with only letters,numbers,etc using the grammar compiler. To be used specifically for this purpose. Even better would be causing the voice recognition to only use the mini .xml file when triggered in this way... and use the normal grammar file if invoked any other way. Get my drift?

                    Could even use PowerVoice from 0l33l (i tihnk it was). All I really want is a prog that writes what I speak into the text box.

                    Comment


                    • #11
                      @Profit
                      sorry man, all i know is a little bit of Autoit, i'm lost in Vb.net, but i see you have it working by now, hope to see that secret program soon.

                      @iceman
                      i need to play more with voice control, my end gives me errors when enabled, cut be my end(nlite) so yea i can help you out with this, but if you know the commands let me know and we can get it to work for sure.
                      BASSRR ==> [Audio Player/MusicDB/Cd Player/Ripper/Rate/Internet Radio/Audio Capture/Youtube Video]
                      RRTube ==> [View/Download Youtube Videos]
                      WifiMan ==> [Wifi Manager]
                      RRec ==> [Audio Capture]

                      If You Want To Be Given Everything, Give Everything Up ...
                      Bird York -- In The Deep --

                      Comment


                      • #12
                        Just gotta find the rr.execute() commands needed for:

                        a) Pausing Audio - $rr.Execute("Pause")
                        b) Switching the voice control .xml file to the mini version (optional) - ?
                        c) Sending the HOTKEY to RR/Voicecontrol application - ? Cant seem to get $rr.Execute("SENDKEY;{F10}") to work
                        (turning the Mic on)
                        d) Sending the ENTER key to RR's OSK/TextBox - $rr.Execute("Enter")
                        (submitting the text to RR and turning the mic off)

                        ....If I use $rr.Execute("Enter") inside the playlist search in BMV2 it works nicely, howver if I use it inside CDRIP to name the Artist/Album/etc, $rr.Execute("Enter") takes me to the main menu.
                        Is there any other way to trigger the enter command?

                        f) Resume audio - $rr.Execute("Resume")

                        Comment


                        • #13
                          a)done i guess.
                          b)well dont know how voice uses this xml, but if is just mater of changing the file, file move would work nicely.
                          c)
                          Code:
                          $Voice = WinGetHandle("RoadRunner", "")
                          WinActivate($Voice, "")
                          Send("{F10}")
                          d)dont know about this one.
                          e)??
                          f)what about if music is paused before all this process? if you use resume or play will start playing no matter what, when finish, maybe use
                          Code:
                           $Status = $rr.getinfo("STATUS")
                          to poll the status info, and according to that, resume or not after this process.
                          BASSRR ==> [Audio Player/MusicDB/Cd Player/Ripper/Rate/Internet Radio/Audio Capture/Youtube Video]
                          RRTube ==> [View/Download Youtube Videos]
                          WifiMan ==> [Wifi Manager]
                          RRec ==> [Audio Capture]

                          If You Want To Be Given Everything, Give Everything Up ...
                          Bird York -- In The Deep --

                          Comment


                          • #14
                            Oops.. there was no (e) . I'm a little overzealous to get this working I guess
                            Good thinking regarding the status polling. Fixed it now.

                            Ive figured (d) out. When you click the text cursor (for 0.8 seconds in the text field) to trigger the Voice recognition to turn on, the process of sending the F10 key is actually taking focus away from the RR CDRIP app. This causes the LOADING screen to be seen. Pressing ESC goes back to the main menu and the entire app is still running correctly (thinking that it has been told to stop voice recognition etc).

                            Test: I replaced the send F10 to RR with send "5" to Calculator, the app worked perfectly and CDRIP DID NOT lose focus from the OSK screen.

                            [All testing was done on BMV2.]

                            Any other ways to trigger the Voice recognition plugin to ON?
                            Can the plugin be recoded to include this option?
                            I would almost prefer it if the plugin wasnt waiting for a hotkey, purely turning the Mic input ON/OFF.

                            I tried the following commands to send the F10 key - all result in LOADING screen:

                            1. ControlSend("RoadRunner","","","{F10}")

                            2. $Voice = WinGetHandle("RoadRunner", "")
                            WinActivate($Voice, "")
                            Send("{F10}")

                            Anyway, heres my current code:
                            Code:
                            #Include <Misc.au3>
                            AutoItSetOption("WinTitleMatchMode", 3)
                            $test = "test"
                            $SpeechOn = -1
                            $PlayStatus = -1
                            If WinExists($test) Then Exit 
                            AutoItWinSetTitle($test)
                            
                            $rr=ObjCreate("RoadRunner.sdk")
                            if not IsObj($rr) then Exit
                            $oMyError = ObjEvent("AutoIt.Error","MyErrFunc")
                            
                            HotKeySet("!{esc}", "_exit") ;for testing only, exit ALT+ESC disable when no need for it.
                            
                            While 1
                            	If  $oMyError.windescription then 
                            		if Not WinExists("RoadRunner") then ExitLoop ;exit if RR is not Running.
                            	EndIf
                            	
                            	If MouseGetCursor() = 5 And _IsPressed("01") then
                            		;Mouse has been clicked where the User wants to dictate
                            		If ($rr.getinfo("STATUS") = "Play") Then	;Check the previous play/pause/stop status
                            			$rr.Execute("pause") ;Pause music		;if playing, then pause the music, else skip
                            			$PlayStatus = 1
                            		EndIf
                            		
                            		;ControlSend("RoadRunner","","","{F10}") ; this does the same as the 3 code lines below - but is slightly less obtrusive
                            		;the above line tries to tell the Voice Recog plugin to start accepting dictation
                            		
                            		$Voice = WinGetHandle("RoadRunner", "") ;trigger hotkey line 1
                            		WinActivate($Voice, "");trigger hotkey line 2
                            		Send("{F10}");trigger hotkey line 3
                            		
                            		$SpeechOn= 1 ; Tells the app that it is about to receive speech commands
                            	EndIf
                            	
                            Sleep (800)                ;push and hold for 0.8 seconds to engage Voice control
                            	
                            
                            	While (MouseGetCursor() = 5 And $SpeechOn = 1) Or (Not(_IsPressed("01")) And Not(MouseGetCursor() = 5) And $SpeechOn = 1)  
                            		;Above line checks that EITHER:
                            		;a)The cursor is a text cursor (Ibeam) and it doesnt care if its being left clicked; or
                            		;b) The cursor is NOT a text cursor (Ibeam) but thats OK, as long as the user isnt left clicking on anything - purely moving the mouse around!
                            		
                            		; While we are waiting for the speech input, we make sure that the cursor stays as a TEXT CURSOR and is in speech mode
                            		; Just sit here and do nothing. This means the app will wait till the 
                            		; ..user finishes talking and issues their "ENTER" command
                            		
                            	WEnd
                            	
                            	If Not(MouseGetCursor() = 5) And $SpeechOn = 1 Then ;As soon as the user gives the ENTER command - This needs work!
                            		;Place code here for
                            		;turn mic off /hotkey - mic has auto-off it seems
                            		$rr.Execute("ENTER") ;press enter/submit text - doesnt always work
                            		If ($rr.getinfo("STATUS") = "PAUSE") AND ($PlayStatus = 1) Then ;at the top we checked if RR was playing/paused/stopped, if playing this resumes, else skips over
                            		$rr.Execute("resume") ;resume audio
                            		$PlayStatus = -1
                            		EndIf
                            	
                            		$SpeechOn = 0 ; Turn off voice commanding - ignore incoming commands/turn mic off
                            		
                            	EndIf
                            	
                            			
                            WEnd
                            ;==============================================================================================
                            ;AI's error reporting Object
                            ;==============================================================================================
                            
                            Func MyErrFunc()
                                Local $err = $oMyError.number
                                If $err = 0 Then $err = -1
                                SetError($err) 
                            Endfunc
                            
                            Func _exit()
                            Exit
                            EndFunc

                            EDIT: changed code to use the '$voice' method (simply changed commenting), as method 1 didnt always work (ControlSend("RoadRunner","","","{F10}"))


                            Ice

                            Comment


                            • #15
                              Voice Dictation

                              @ICE

                              Any other ways to trigger the Voice recognition plugin to ON?
                              Can the plugin be recoded to include this option?
                              I would almost prefer it if the plugin wasnt waiting for a hotkey, purely turning the Mic input ON/OFF.

                              I agree with turning the mic on/off. I had to switch to using the hotkey because the mic was always on and the voice control was in listen mode. I noticed that outside noise or conversation that may have resembled commands would get executed.

                              When I left work last night I tested what I was talking about. In order to type letters using the current voice commands the system had to be prompted for every letter - press a button letter "a" press a button letter "b" and so on. IMO the voice command system needs to be more like a Dragon Dictate software or the Microsoft voice dictation. Then the system could disseminate entire words or phrases. This would also facilitate the use of playing artists, or albums on command. I know there is a plugin (SmartPlaylist) that does this but if the voice commands plugin can be "re-written" may want to consider that as well.

                              Comment

                              Working...
                              X