Thread: Telematics.net

I'm thinking of a media base class (or interface). Video, Audio, Picture, or Document classes could implement (or inherit) media. WMA, Ogg, MP3, ... then could mplement/inherit Audio while JPG, BMP, TIF, ... might implement/inherit Picture. The Audio class might also implement/inherit and ID3TAG type. Obviously you have to think this through since .Net doesn't support multiple inheritance though it does allow a class to implement multiple interfaces.

The Player class would then consume Media objects but this is where the convenience of your method starts to reveal itself. If a Media objects know how to play itself then there is no need for the Player class to discover the derived type(s) of each media. This would be convenient. On the other hand, if each class derived from media must implement its own play method, then that might dirty their implementation with player specific implementation details.

thats a great idea, I'll actually be posting my audio class soon ! (still gotta code it, but I have the psuedo all ready)

just save the ID of the file playing in your settings db and its position instead of making each song say playing or pause or whatever. its more effiecient this way.

i'm interested in how you made the voice reognition stuff? because with like navivoice you have to predefine words

I don't think you have to predefine anything to use SAPI. Of course, the quality of recognition goes up if you prepare a profile (via training) that matches the speaker and environment. Even more important is providing a context sensitive grammar. It's not as tough to work with as you might think.

With sapi and a song's list I've found that its much better to use a "pre-defined list" (i.e. the name of the mp3)

artist - songname.mp3 (here's where it gets tricky) I had to seperate songname from artist using the mp3 filename or the ID3 tags. (some songname's arent named correctly, or they dont follow any sort of format) Only song's that followed the format would be entered into the XML list. (made at program's runtime. after a new song has been found.)

From here I had a working list to work from Artists and Songname's ( I made them seperatly)

Now I added this XML file to sapi's engine, without using its default "dictionary" So SAPI only knew the artists and song's. As you may know songname's can be a non-english word so this is why I gave SAPI a list of formatted songnames so it could phenetically guess the best choice.

You can say: "play megalomaniac" or "play megalomaniac by incubus" or "show incubus"

From here SAPI kicks my application back what it heard or what it thought it heard. This is where I did some more coding to get better accuracy (without training SAPI)

I broke down the string it heard: "megalomaniac" then I compare it to song's in the database with a custom made alogrithym if the system finds a hit with a percentage from the database greater than 65 it logs it and keeps searching if it doesnt find anything better, it chooses that song if no songs hit greater than 65 then it doesn't play a song.

There is obviously ways to improve this system, but I've tested the code (which I can post, its in VB6) and it actually chose the correct song 97% of the time when it had to "guess" what the user said.

Finally. why did I do this ? A vehicle's envirnment can be noisey so if you verbally choosing a song "play song megalomaniac" and someone suddenly cuts you off and the songname turns into megalo****offasshole ! SAPI would disregard the statement and no song would be chosen.

It's almost like giving SAPI an intellagent feature to "guess" the best possible song. (SAPI does a guess under the event hypothesis)

Sounds cool! You could extend that to give you a pop-up (or voice) showing a list of potential matches. This way you can drill down quickly.

Ok, Here it is (it's not commented well) but again, its all prototyping.

Code:

Dim current_volumeLevel As Integer
Dim fso As New FileSystemObject
Static say_again As Integer

Dim hypoth As String
Dim hypoth_str_state As String
Dim b As Integer

Dim str_data As String
Dim pntr As Integer
Dim c As Integer
Dim d As Integer
Dim data_array1(50) As String
'Dim data_array2(10) As String


Dim data_array1Size As Integer
Dim data_array2Size As Integer
Dim var1 As String
Dim var2 As String
Dim x2 As Integer
Dim x3 As Integer

Dim var3 As String
Dim compentcy_score As Integer
Dim percentage_correct As Double

Dim compare As Double
Dim place_holder As Integer




current_volumeLevel = media_player.wmpplayer.settings.Volume
media_player.wmpplayer.settings.Volume = 0
' mute media player volume

begin_mp3Select:
If fso.FileExists(App.Path & "/grammer/mp3grammer.xml") Then
    
    
    If say_again > 0 Then
        speak ("Say that again ?")
    Else
        speak ("What shall I do ? ")
    End If
    
    
    
    media_search.myGrammar.CmdLoadFromFile App.Path & "/grammer/mp3grammer.XML", SLODynamic
    
    
    'this statement plays the beep, loads the grammer file and activates recognition

Else
    'If grammer file is not created, create it then call this function over again.
    Call write_XML_MP3Grammer
    GoTo begin_mp3Select:
    
End If


Dim initial_time As Double
Dim current_time As Double
Dim duration_time As Double

initial_time = Timer
media_search.reco_end = False

'waits 3 seconds to listen to command

media_search.myGrammar.CmdSetRuleIdState 0, SGDSActive
Do While media_search.txt_search.text = ""
    DoEvents
    current_time = Timer
    duration_time = current_time - initial_time
    
    If duration_time > 3.5 Then
        GoTo done:
    End If
    
Loop

done:

initial_time = Timer

Do
    current_time = Timer
    duration_time = current_time - initial_time
    DoEvents
Loop Until media_search.reco_end = True Or duration_time > 2.5



media_search.myGrammar.CmdSetRuleIdState 0, SGDSInactive

'inactivate voice recognition


'Process information

Dim primary_detection As Boolean
Dim recognition_string As String


If media_search.txt_search.text = "" Then
    primary_detection = False
    recognition_string = media_search.release_hypothesisPhrase
Else

    primary_detection = True
    recognition_string = media_search.txt_search.text
    media_search.txt_search.text = ""
End If


If InStr(recognition_string, "by") Then
    If InStr(recognition_string, "play") Then
        voiceCommand = command_songby
        GoTo SelectCommand:
    End If
    
End If

If InStr(recognition_string, "play") Then
    voiceCommand = command_song
    GoTo SelectCommand:
End If

If InStr(recognition_string, "Show artist") Then
    voiceCommand = command_showArtist
    GoTo SelectCommand:
Else

    voiceCommand = command_null
End If





SelectCommand:
'recognition_string = holds the data to be processed.


Dim command_playby_bypos As Integer
Dim songname As String
Dim artist As String
Dim i As Integer
Dim X As Integer
Dim playsong As Boolean
Dim str1 As String
Dim str2 As String


Select Case (voiceCommand)

    Case (command_null)
        ' No command was initiated
        If say_again = 1 Then
            speak ("No command was heard.")
            say_again = 0
            Exit Sub
        Else

            say_again = say_again + 1
            Call Voice_mp3_select
        End If
        
    Case (command_songby)
        ' "play & by"
        command_playby_bypos = InStr(1, recognition_string, "by")
        songname = Mid(recognition_string, 1, command_playby_bypos - 1)
        songname = Right(songname, Len(songname) - 4)
        artist = Mid(recognition_string, command_playby_bypos + 2, Len(recognition_string))
        
        'From recognition
        songname = Trim(songname)
        artist = Trim(artist)
        
        
        
        If primary_detection = True Then
            For i = 0 To media_search.mp3_list.ListCount
            
            If InStr(1, LCase(media_search.mp3_list.List(i)), artist) Then
                If InStr(1, LCase(media_search.mp3_list.List(i)), songname) Then
                playsong = True
                GoTo playthesong:
                End If
            End If
            
            Next i
        Else
        'working with a hypothesis statement here
        '80's - Jefferson Starship - We built this city.mp3
  
        

        End If
        
        
        
        
    Case (command_song)
        'play
        songname = Mid(recognition_string, 5, Len(recognition_string))
        songname = Trim(songname)
        
        If primary_detection = True Then
        
            For i = 0 To media_search.mp3_list.ListCount
                If InStr(1, LCase(media_search.mp3_list.List(i)), songname) Then
                    playsong = True
                    GoTo playthesong:
                End If
            Next i
        Else
        'Working off of a hypothesis statement
        
            'Scan list for instring of the songname
            For i = 0 To media_search.mp3_list.ListCount
                If InStr(LCase(media_search.mp3_list.List(i)), songname) Then
                    data_array1(c) = media_search.mp3_list.List(i)
                    c = c + 1
                End If
            Next i
            
            
            
            'process the data
            For i = 0 To c - 1
                x2 = 1
                Do
                    var2 = Mid(data_array1(i), x2, 1)
                    If var2 = "-" Then
                        x3 = x2
                    End If
                    x2 = x2 + 1
                Loop Until x2 = Len(data_array1(i))
                
                
                
                var2 = Mid(data_array1(i), x3 + 1, Len(data_array1(i)))
                var2 = Left(var2, Len(var2) - 4)
                
                'var2 is = confessions pt2
                'MsgBox var2 & vbCrLf & hypoth
                var2 = LTrim(var2)
                
                x2 = 1
            


            Do
                var1 = Mid(var2, x2, 1)
                
                If x2 > Len(songname) Then
                    'compentcy_score = compentcy_score + 1
                Else
                
                    var3 = Mid(songname, x2, 1)
                    
                    If LCase(var3) = LCase(var1) Then
                        compentcy_score = compentcy_score + 1
                    End If
                   
                End If
                x2 = x2 + 1
            Loop Until x2 = Len(var2)
            
            percentage_correct = (compentcy_score / Len(var2))
            
            If percentage_correct > 0.6 Then
                If percentage_correct > compare Then
                    compare = percentage_correct
                    place_holder = b
                End If
            End If
            
                        
        Next i
        
        
        
    
        'media_search.lstmenuitems.Clear
        For i = 0 To media_search.mp3_list.ListCount
            If data_array1(place_holder) = media_search.mp3_list.List(i) Then
                playsong = True
                GoTo playthesong:
            End If

        Next i
                
                
                
                
                
                
                
            
            
            
        
        End If
        
        
    Case (command_showArtist)
    
    Dim showArtist(100) As String
    
        'Show Artist
        artist = Mid(recognition_string, 13, Len(recognition_string))
        artist = Trim(artist)
        
        
        
        If primary_detection = True Then
        
        
        For i = 0 To media_search.mp3_list.ListCount
            If InStr(1, LCase(media_search.mp3_list.List(i)), artist) Then
                
                If X < 100 Then
                    showArtist(X) = Return_Mp3(media_search.mp3_list.List(i))
                    X = X + 1
                End If
                
                
                
            End If
        Next i
        
        'MsgBox showArtist(1)
        
        
        Else
        
        'Working off of a hypothesis statement
        End If
        
        
        
        GoTo showArtistC:
End Select




showArtistC:





If voiceCommand = 0 Then
    media_player.wmpplayer.settings.Volume = current_volumeLevel
    Exit Sub
End If


If voiceCommand > 0 Then
    say_again = 0
End If

media_player.wmpplayer.settings.Volume = current_volumeLevel


If showArtist(b) = "" Then
        speak ("Artist not found")
        Exit Sub
End If

media_list.lstmenuitems.Clear

Dim additem As Boolean

    
 For b = 0 To X - 1
    media_list.lstmenuitems.additem (Return_Mp3(showArtist(b)))
 Next b
 
    
    For i = 0 To media_search.mp3_list.ListCount - 1
    
    str1 = Return_Mp3(media_search.mp3_list.List(i))
    
        For b = 0 To X - 1
                
            str2 = showArtist(b)
            
            If Not str1 = str2 Then
                additem = True
            Else
                additem = False
                b = X - 1
            End If
            
            
        Next b
        
        If additem = True Then
            media_list.lstmenuitems.additem str1
        End If
        
    
    Next i

This simply plays the song, but shown here just so you all get see some more code.

Code:

playthesong:

If voiceCommand > 0 Then
    say_again = 0
End If

    
media_player.wmpplayer.settings.Volume = current_volumeLevel

If playsong = True Then
    
    If media_search.mp3_list.List(i) = "" Then
        speak ("Song not found")
        Exit Sub
    End If
    
    media_list.lstmenuitems.Clear
    media_list.lstmenuitems.additem media_search.mp3_list.List(i)
    media_search.mp3_list.ListIndex = i
    
    For X = 0 To media_search.mp3_list.ListCount - 1
        If Not Return_Mp3(media_search.mp3_list.List(X)) = Return_Mp3(media_search.mp3_list.List(i)) Then
            media_list.lstmenuitems.additem media_search.mp3_list.List(X)
        End If
    Next X
    
    setSongname_change (True)
    media_list.lstmenuitems.ListIndex = 0
    media_state = media_play
    Media_control
    
End If
Exit Sub

Hmm...maybe time to start packaging these code snippets into actual projects that people can compile and play with?