Using the Voice Chat Function

zepeto voice chat preview version package enables voice chat in the world install zepeto voice chat and set the mode 1\) install window → package manager → zepeto voice chat this feature is available in a world where multiplayer elements are implemented testing is only available in the mobile test, not editor test due to ios policy, the device volume does not drop to zero when the microphone is turned on in worlds with voice chat enabled the device volume will drop to zero when the microphone is turned off 2\) click the \[▼] button in the middle menu of the unity project screen that's under development click the \[open world settings] button in the pop up menu that appears 3\) if the zepeto voice chat package is installed correctly, you will see the voice chat mode option added 4\) in voice chat mode, please set the desired mode basic use the voice chat feature provided by default script use the voice chat feature by implementing the voice chat api you can process room access entry/exit for voice chat at runtime implement voice chat for distinct teams (e g , red team voice chat and blue team voice chat) detecting current speaking user basic mode using the voice chat feature a world with basic mode voice chat looks like this you can also create more entertaining voice chats using the voice modulation feature voice chat is automatically enabled when entering the world with voice chat function 👍 the first time you use the voice chat feature in zepeto, a pop up regarding microphone access will appear you must allow access in order to use the voice chat function you can use it by touching the voice chat button in zepeto app 3 26 000 and above, the voice chat speaker icon does not appear if you want to make the speech icon appear, please change to script mode and follow the guide to write your own code script mode using the voicechat api script mode enables voice chat in channels separated by teams please refer to the function description and example below to implement the feature api description oninitializedevent the voice chat apis are available after a true event occurs in the initialization event if this event is true, voice chat is available, and if it is false, voice chat is not available enterroom(roomproperty roomproperty) functions to enter a voice chat room \ after successfully entering the voice chat room, the voice chat button will appear exitroom() function to leave a voice chat room onroomconnectedevent an event to check the status of entering a voice chat room if this event is true, the user has entered the voice chat room; if it is false, the user has left the voice chat room onspeechdetectedevent\<string, boolean> event to detect the user who is currently speaking in the voice chat \ string is the userid of the speaking user \ boolean is true when the utterance starts, false when the utterance ends changeteamid(number teamid) function to change the teamid used in the voice chat room \ can only be set to an int value of 1 or more roomproperty class api description setaudiomode audio mode to use in voice chat \ audiomode omnidirectional voice chat mode with no voice attenuation based on distance (you can use it with or without the zepeto character) \ audiomode directional voice chat mode with voice attenuation based on distance (the zepeto character must be present in the scene, as this mode works based on the zepeto character's location) setteamid teamid to be used in voice chat to enable voice chat between users with the same teamid \ can only be set to an int value of 1 or higher example of team specific voice chat implementation here is some example code to try out the voice chat api 1\) add a button and log text to the canvas as shown in the image below to enter the voice chat team channel and display the status log button button blueteam blue team voice chat entry button button redteam red team voice chat entry button button exit exit voice chat button text text log text to display the chat log text team text to display the list of members of the same voice chat team channel text currentspeaking text to display the userid who is currently speaking in the voice chat 2\) create project > create > zepeto > typescript and rename it to voicechattest 3\) write a sample script as shown below voicechattest import { button, inputfield, text } from 'unityengine ui'; import { zepetoscriptbehaviour } from 'zepeto script' import { audiomode, roomproperty, voicechatcontroller } from 'zepeto voice chat' export default class voicechattest extends zepetoscriptbehaviour { // declare ui components as public members public blueteambutton button; public redteambutton button; public exitroombutton button; public logtext text; public currentteamidtext text; public currentspeakinguseridstext text; // declare private members to store speaking user ids and team id input field private currentspeakinguserids string\[]; private teamidinputfield inputfield; start() { this currentspeakinguserids = new array(); // add event listeners for button clicks this blueteambutton onclick addlistener(()=>{ this enterblueteam(); }); this redteambutton onclick addlistener(()=>{ this enterredteam(); }); this exitroombutton onclick addlistener(()=>{ this exitvoicechatroom(); }); // add event listeners for voice chat controller voicechatcontroller oninitializedevent addlistener(init => this oninitialized(init)); voicechatcontroller onroomconnectedevent addlistener(connected => this onroomconnected(connected)); voicechatcontroller onspeechdetectedevent addlistener((userid, speechdetected) => this onspeechdetected(userid, speechdetected)); this logtext text = "\[voicechat] voicechattest start"; } // ondestroy method called when the script is destroyed ondestroy() { voicechatcontroller oninitializedevent removealllisteners(); voicechatcontroller onroomconnectedevent removealllisteners(); voicechatcontroller onspeechdetectedevent removealllisteners(); } // method called when the voice chat system is initialized private oninitialized(initialized boolean) { this logtext text = "\[voicechat] oninitialized " + initialized; } // method called when the voice chat room is connected private onroomconnected(connected boolean) { this logtext text = "\[voicechat] onroomconnected " + connected; } // method called when a user's speech is detected or not detected private onspeechdetected(userid string, speechdetected boolean) { this logtext text = "\[voicechat] onspeechdetected " + userid + ", " + speechdetected; if (speechdetected) { this currentspeakinguserids push(userid); } else { let index = this currentspeakinguserids indexof(userid); this currentspeakinguserids splice(index, 1); } let userids = "\[currentspeakinguserids]\n"; for (let i = 0; i < this currentspeakinguserids length; i++) { userids += this currentspeakinguserids\[i]; userids += "\n"; } this currentspeakinguseridstext text = userids; } // method to enter the blue team's voice chat room public enterblueteam(){ this entervoicechatroom(2); } // method to enter the red team's voice chat room public enterredteam(){ this entervoicechatroom(3); } // method to enter a voice chat room based on the given team index private entervoicechatroom(teamindex number) { console log("\[voicechat] entervoicechatroom"); // create a new roomproperty object and set its properties let roomproperty = new roomproperty(); roomproperty setaudiomode(audiomode omnidirectional); roomproperty setteamid(teamindex); voicechatcontroller enterroom(roomproperty); this currentteamidtext text = "teamid " + roomproperty teamid tostring(); } // method to exit the voice chat room public exitvoicechatroom() { this logtext text = "\[voicechat] exitvoicechatroom"; voicechatcontroller exitroom(); } // method to change the team id for the current user public changeteamid() { // get the new team id from the input field and parse it as an integer let teamid = parseint(this teamidinputfield text); this logtext text = "\[voicechat] changeteamid " + teamid; this currentteamidtext text = "teamid " + teamid tostring(); // change the team id in the voice chat system voicechatcontroller changeteamid(teamid); } } code description when the script runs, it will register an event on each button in the start() function and register the oninitialized, onroomconnected, and onspeechdetected events on the voicechatcontroller when the blue team button or red team button is clicked, the entervoicechatroom() function is executed and the user enters the voice chat room by entering the specified roomproperty the room properties in the example are as follows roomproperty setaudiomode audiomode omnidirectional the audio mode omnidirectional is a mode where the volume sounds the same regardless of the player's position in contrast, directional mode is a 3d sound mode, which means that the volume will sound different depending on the character's position, so the farther away the character is, the lower the volume will be roomproperty setteamid channel 2 for blue team, channel 3 for red team setteamid acts like a channel in voice chat a team that enters channel 2 can only voice chat with players in channel 2, and a team that enters channel 3 can only voice chat with players in channel 3 exitvoicechatroom() will exit the current voice chat team channel onspeechdetected() will output the player id that is currently using voice chat in the team channel to the log text 4\) after completing the script, assign the button and text created in step1 in the inspector 5\) run the test on mobile via qr code or test link, and you can see that the voice chat activation button is created when the enter team button is pressed, and whenever a player speaks, the log shows which player is in voice chat, as shown in the following screen example of displaying a voice chat speech bubble image in zepeto app 3 26 000 and later versions, the voice chat utterance icon does not appear if you want to handle the utterance icon to appear, please follow the guide and write your own code 1\) to show a speech bubble above the character's head when the player is in voice chat, you will need to create a prefab that uses a speech bubble image import the speech bubble png image you want to use as shown below and change its texture type to sprite 2\) project > create > prefab and rename it to chatbubble 3\) double click the chatbubble prefab to enter the prefab editing mode select the chatbubble prefab and choose transform > position > change the y value to 0 35 4\) create a canvas inside the prefab object double click the chatbubble prefab to enter prefab edit mode, and then add hierachy > ui > canvas change the values of the recttransform component of the canvas to the following posx 0 , posy 0 width 100, height 100 scalex 0 005, scaley 0 005, scalez 0 005 change the rendermove of the canvas component to world space 5\) create a speech bubble image inside the canvas add hierarchy > ui > image as a child of canvas, and rename it chatbubbleimage change the values of the recttrasform component of the chatbubbleimage to the following width 42, height 42 register the image sprite you imported in step 1 as the source image in the image component 👍 tip you can also animate the speech bubble image sprite you can also use separate bubble images for different teams 6\) project > create > zepeto > create a typescript and rename it to voicechatbubblecontroller 7\) write a sample script as shown below voicechatbubblecontroller import { gameobject, object } from 'unityengine'; import { knowsockets, zepetoplayers } from 'zepeto character controller' import { zepetoscriptbehaviour } from 'zepeto script' import { audiomode, roomproperty, voicechatcontroller } from 'zepeto voice chat'; export default class voicechatbubblecontroller extends zepetoscriptbehaviour { // public gameobject variable to store the voice chat bubble prefab public voicechatprefab gameobject; // private map to store gameobjects associated with user ids private voicebubblemap map\<string, gameobject> = new map\<string, gameobject>(); start() { // add event listeners for voice chat controller voicechatcontroller oninitializedevent addlistener(init => this oninitialized(init)); voicechatcontroller onroomconnectedevent addlistener(connected => this onroomconnected(connected)); voicechatcontroller onspeechdetectedevent addlistener((userid, speechdetected) => this onspeechdetected(userid, speechdetected)); } // method called when the voice chat system is initialized private oninitialized(initialized boolean) { console log("\[voicechat] oninitialized "); this entervoicechatroom(1); } // method called when the voice chat room is connected private onroomconnected(connected boolean) { console log("\[voicechat] onroomconnected "); } // method to enter a voice chat room based on the given team index private entervoicechatroom(teamindex number) { console log("\[voicechat] entervoicechatroom"); // create a new roomproperty object and set its properties let roomproperty = new roomproperty(); roomproperty setaudiomode(audiomode omnidirectional); voicechatcontroller enterroom(roomproperty); } // method called when a user's speech is detected or not detected private onspeechdetected(userid string, speechdetected boolean) { console log("\[voicechat] onspeechdetected " + userid + ", " + speechdetected); // check if the user id is not in the voice bubble map and create a voice bubble if it is not if (!this voicebubblemap has(userid)) { this createvoicebubble(userid); } this setvoicebubble(userid, speechdetected); } // method to set the active state of the voice bubble for a given user id private setvoicebubble(userid string, speechdetected boolean) { const chatbubble = this voicebubblemap get(userid); chatbubble setactive(speechdetected); } // method to create a voice bubble for a given user id private createvoicebubble(userid string) { // get the head socket of the user's character const headsocket = zepetoplayers instance getplayerwithuserid(userid) character getsocket(knowsockets head upper); // instantiate the voice chat bubble prefab at the head socket position const instancebubble = object instantiate(this voicechatprefab, headsocket) as gameobject; // add the instantiated bubble to the voice bubble map this voicebubblemap set(userid, instancebubble); instancebubble setactive(false); } lateupdate() { // check if the voice bubble map is empty and return if it is if (this voicebubblemap size === 0) { return; } // iterate through the voice bubble map and update the rotation of each bubble gameobject this voicebubblemap foreach((bubbleobject gameobject) => { // set the rotation of the bubble object to match the camera's parent transform rotation bubbleobject transform rotation = zepetoplayers instance zepetocamera cameraparent transform rotation; }); } } code description when the script is executed, it will register events on each button in the start() function and oninitialized, onroomconnected, and onspeechdetected events on the voicechatcontroller after oninitialized(), the voice chat button will be exposed on the player's screen when they enter the world, and when they press the button, entervoicechatroom() will be executed to enter the voice chat room whenever the player activates voicechat and speaks, onspeechdetected() is executed, and if speechdetected is tue, the bubblechat object is enabled, and if false, it is disabled the first time the onspeechdetected() function is activated, createvoicebubble() will instantiate a voicechatprefab gameobject to float above the player's head and register it as map data in voicebubblemap we use lateupdate() to update the rotation of the voice chat bubble every frame to match the cameraparent of the zepeto world this ensures that the speech bubble image is always facing the camera it first checks to see if voicebubblemap is empty, otherwise it updates the rotation of the speech bubble image prefab 8\) after finishing writing the script, assign the voicechatprefab entry in the inspector as the speech bubble prefab you created in step 1 9\) run it on mobile via a qr code or test link and you should see the speech bubble image appear above the character's head whenever the player uses voice chat voice modulation starting with voice chat version 0 2 1 preview, you can modulate the voice chat voice in script mode https //www youtube com/watch?v=yxfpuqypbqk https //www youtube com/watch?v=yxfpuqypbqk try voice modulation using the api guide and the sample code voicetype (enum) value description type00 0 original type01 1 chipmunk type02 2 uncle type03 3 echo type04 4 deep voice type05 5 robot type06 6 dialect type07 7 megaphone type08 8 beast type09 9 machine type10 10 strong current type11 11 kid type12 12 hedgehog here is a sample script of voice modulation import { button } from 'unityengine ui'; import { zepetoscriptbehaviour } from 'zepeto script'; import { roomproperty, voicechatcontroller, voicetype } from 'zepeto voice chat'; export default class voicemodulationmanager extends zepetoscriptbehaviour { // buttons for setting different voice types public buttons button\[]; start() { const voicetypes = \[ voicetype type00, voicetype type01, voicetype type05, voicetype type03, ]; for (const \[index, button] of this buttons entries()) { button onclick addlistener(() => { // set the voice type corresponding to the clicked button index voicechatcontroller setvoicetype(voicetypes\[index]); }); }; // method called when the voice chat system is initialized voicechatcontroller oninitializedevent addlistener( init => { voicechatcontroller enterroom(new roomproperty());; } ); // method called when the voice chat room is connected voicechatcontroller onroomconnectedevent addlistener( connected => { // set the initial voice type and activate loopback voicechatcontroller setvoicetype(voicetypes\[0]); voicechatcontroller enableloopback(true); } ); } // method to exit the voice chat room ondestroy() { voicechatcontroller oninitializedevent removealllisteners(); voicechatcontroller onroomconnectedevent removealllisteners(); } } code description adds an onclick listener for each button to set different voice modulation types elements of the same index in the buttons array and the voicetypes array are matched upon entering the world, voicechatcontroller is initialized, and the room connection is established automatically the initial voice modulation type is set to type00 , and loopback is enabled to hear your own voice note that for the invocation of setvoicetype and enableloopback , a room connection is necessary hence, the initial setup is handled within the listener of the onroomconnectedevent change the local player's audio playback location you can change the local player's voice chat location using voicechatcontroller setlocalplayertransform() using this, you can create various scenarios by setting your location to a specific space or object location when speaking in voice chat 📘 tips the voice chat’s transform is linked to the local player’s zepeto character transform therefore, if the local player’s zepeto character hasn’t been created in the world, your voice may not be transmitted in this case, you can use voice chat by setting the voice playback location within the current scene using setlocalplayertransform() api description public static setlocalplayertransform($transform unityengine transform)\ void function to change the local player's audio playback position ❗️ caution setlocalplayertransform() must be called sometime after voicechatcontroller oninitializedevent(true) when voice chat is initialized 1\) below is a sample script for the voice position change function voicetransformsample import { gameobject, transform, vector3 } from 'unityengine'; import { button } from 'unityengine ui'; import { zepetoscriptbehaviour } from 'zepeto script'; import { audiomode, roomproperty, voicechatcontroller } from 'zepeto voice chat'; export default class voicetransformsample extends zepetoscriptbehaviour { // button to set voice transform close public buttonclosertransform button; // button to set voice transform far public buttonfarthertransform button; // transform for the voice position private voicetransform transform; start() { // create new gameobject assign the transform to store voice transform object const voicetransformobject = new gameobject; this voicetransform = voicetransformobject transform; // method called when the voice chat system is initialized voicechatcontroller oninitializedevent addlistener( init => { let roomproperty = new roomproperty(); // set audio mode to directional 3d space audio roomproperty setaudiomode(audiomode directional); voicechatcontroller enterroom(roomproperty); // set the voice chat transform for local player's voice voicechatcontroller setlocalplayertransform(this voicetransform); } ); // set voice transform position to (0,0,0) when 'buttonclosetransform' is clicked this buttonclosertransform onclick addlistener(()=>{ this voicetransform position = vector3 zero; }) // set voice transform position to a distant point when 'buttonfartransform' is clicked this buttonfarthertransform onclick addlistener(()=>{ this voicetransform position = new vector3(0,0, 10); }) } // method to exit the voice chat room ondestroy() { voicechatcontroller oninitializedevent removealllisteners(); voicechatcontroller onroomconnectedevent removealllisteners(); } } code description when the scene starts, create a game object called voicetransformobject in the start() function and register the transform in voicetransform when the voice chat oninitialized listener is called, please apply the following settings set voice chat to 3d spatial audio mode via setaudiomode(audiomode directional) set the local player's voice chat position to voicetransform through voicechatcontroller setlocalplayertransform() buttonclosertransform sets the position of voicetransform to the origin (0, 0, 0) when the button is clicked buttonfarthertransform sets the position of the voicetransform to (10, 0, 0) when the button is clicked 2\) register the button on the canvas in the scene to the buttonclosertransform and buttonfarthertransform of the voicetransformsample component 3\) if you run it on your mobile phone using the qr code or test link, you can check the following when you press the closer button, your voice will sound closer to others when you press the farther button, your voice will be heard far away from others https //www youtube com/watch?v=y pj4rcshts https //www youtube com/watch?v=y pj4rcshts