TTS Latency Optimization – we’ve recently come to realize that TTS response times have deteriorated due to increasing levels of usage. Both additional load and accumulated data have had a detrimental effect over time. We’ve just implemented an update to address the problem. With this update response times are now faster than they have ever been. In most use cases you should find that latency has been reduced by over 50%.
JavaScript Framework Examples – We’ve added new framework examples for ReactJS, NextJS, Angular and Vue. This should prove helpful as customers have repeatedly run into issues that were not covered by our initial technical example. We’ve identified a set of five examples that together cover most envisioned scenarios.
The following examples are now available for each framework:
Conversation Example: Multiple avatars, the use of Portals.
AI Text Example: Using the sayAI API with text input
AI Audio Example Using the sayAI API with audio input
All the examples come with full source code, and are available on the SitePal support page.
Access Audio Raw Data – Note: this feature is available to Platinum and Integrator Plan customers.
We’ve added a new API function ‘getAudioObject’ that enables access to the raw audio data as it is being played. To understand why this feature might be useful, let us share the scenario that brought this about.
In an AI Agent implementation our customer preferred to have an active mic at all times to allow users to interrupt the avatar. They used echo cancellation to eliminate avatar speech from the input & required access to the realtime audio data to make this work.
We’re sure customers will find other equally cool and unexpected ways to take advantage of this feature.
Eleven Labs Lip Sync Fix – due to special characteristics of Eleven Labs audios they were not being correctly processed, causing lipsync to be sub-par for Eleven Labs audios. This problem was identified and fixed on Aug 9th.
Also, if you’ve missed it please see our recent announcement on this blog regarding Chatbase Pre-Integration. We’re very excited about it.
Of our future plans & projects perhaps worth mentioning at this time are the following –
Flutter Support – we’re looking to add Flutter support and Flutter code examples to our support materials. This should be available in the next few weeks.
Photoface 3D API – the intention is to enable customers to enable their end users or customers with a Photoface like tool, that end users could use to create & use their own avatars. This feature will be available to Integrator Plan customers only. This project is in the planning phase. Please contact us if you have an interest in this capability.
That’s it for the moment. We hope that you have found this information useful. Please contact us with any comments or questions. We’d love to receive your feedback.
We are delighted to let you know that SitePal is now pre-integrated with Chatbase, one of the leading tools for creating AI chatbots for business.
While this blog post is similar to one we posted several months ago regarding ChatGPT integration, the information here and in the referenced docs is updated and includes new relevant details. Also – this is a different AI Bot solution, with what we believe to be greater potential for businesses to take advantage of.
So what does this mean for you, and how can you take advantage of it?
With Chatbase you can easily create an AI Bot which knows all about your business and products, and then use a SitePal avatar as an Intelligent Agent to communicate with your website visitors. Your visitors could then learn about your products and services by conversing with your SitePal avatar!
And, you can do all that without complex programming! Creating your AI bot with Chatbase and teaching it all about your products, takes only moments (assuming you already have a website and/or documentation). Integrating it with SitePal requires copying your API key from your Chatbase account to your SitePal account. Yes, It’s that easy.
The following technical example allows you to select a Bot to interact with. Select ‘Chatbase’ from the menu & try it out by asking questions. Note: This Inteligent Agent was set up by scanning the SitePal Site and Blog, so it knows quite a bit about SitePal. Ask it!
At the risk of being slightly technical, let us explain what you actually need to do to create an Intelligent Agent implementation, using Chatbase, for your website.
In broad strokes the effort would consist of 2 parts:
1. Open an account with Chatbase and teach your bot all about your products & business by pointing it to your website(s) and/or uploading documents.
2. Setup a SitePal character on your website to answer users’ questions by connecting to your Chatbase bot.
The first part is done by using the simple tools provided by Chatbase. It’s totally straightforward.
The second part is where our Pre-Integration comes into play. To publish your SitePal character as a speaking Chatbase-driven Intelligent Agent, you need to add your Chatbase API key & Bot ID to your SitePal account. Then use our sayAI API call (yes, ONE call !) to have your character speak the responses to user questions.
We did the heavy lifting – so you don’t have to!
Please see detailed step-by-step instructions in the following technical note – and start taking advantage of intelligent agent technology today!
Additional 3rd party AI vendors will be integrated with SitePal soon. If you have a specific one in mind, drop us a note and let us know. We will do our best to prioritize.
Warm Regards,
The SitePal Team
Photoface™ 3D is a SitePal® feature that enables you to create your own 3D character from a photo. To access this feature, open the SitePal Editor, and in the ‘Models’ tab click on ‘Create Your Own’.
Photoface 3D is available with the Gold Plan & higher
In this post we would like to share best practices and tips that should help you make the most of this unique feature. Getting good results with Photoface 3D is partly due to optimal photo selection, and partly to acquired skill with the creation process. We will discuss both.
I. Photo Selection Tips
Your first rule – pick photos that meet our guidelines –
Mouth closed (no teeth showing) – smile is ok
Face looking forward, directly at the camera
High quality, hi-resolution photo (more about this below)
Evenly lit & in focus – no shadows on the face
No eyeglasses, hair or other obstructions over the face. Facial hair should not be a problem.
High hair (a bun or Afro for example) can cause problems and should be avoided. Long flowing hair should not be a problem.
Optimal Resolution
So what is the right resolution to use?
Use a low resolution image – and your character may appear pixelated (grainy) when displayed on your web page. Use too high a resolution – and you’d be adding weight (download size) to your character unnecessarily.
The rule of thumb is roughly 1000 x 1000 pixels for the area immediately surrounding the face (while the photo itself may be larger). There is not much value to be gained by using a higher resolution, because on most screens it is unlikely that that these extra pixels will come into play. But if your implementation has unique viewing characteristics, do the math & decide for yourself.
Image Composition
We suggest to use a photo that displays at least the upper part of the body, not just the head. For best results the body should be visible from below the chest & up. This will allow you to create a character with a large enough portion of the body, which will enable more flexibility when positioning on your web page.
Background – does it matter?
The Photoface 3D process does a pretty good job identifying the persons’ outline & auto-removing the background. But having a complex, noisy background can sometimes cause problems. While we do not require it, if you have the option we encourage the use of a uniform color background in your photo.
Pre-Edit the Image
Being fluent in Photoshop can be an advantage in certain cases. For example – a short neck (or no neck) photo can cause the shoulders to animate when the face moves. Sometimes that works well, and other times not so much. In those cases editing the photo by adding or extending the neck can help resolve.
Another example is the so called “high hair” scenario. In some images, the face might be slightly inclined forward, resulting in the upper scalp occupying a bit more of the vertical length of the face than it normally would. This can result in visual problems when the Avatar animates. Sometimes it can be addressed by adjusting the Camera Angle setting (see below). But it can also be improved by trimming the upper scalp. This phenomena can exacerbated when the face shape is long and narrow.
A long narrow face can result in “high-hair” distortion – slight trim of the upper scalp may help
Special Cases
Note that a photo may not work very well despite satisfying all requirements. Sometimes it is best to simply select a different photo. The opposite is also true – we sometimes see great results with faces that are off center, looking slightly sideways, or wearing eyeglasses.
Photoface 3D technology is optimized to the proportions of a “normal” human face. Faces with unusual proportions may not work very well. For example – an alien face with exceptional large or protruding eyes.
Despite that, we sometimes see customers achieve surprisingly good results with images of inanimate objects. A customer recently shared with us a very lovely “carrot face”, created by anthropomorphing an image of a carrot. We encourage you to use your imagination – but bear in mind that the tool was designed for human facial proportions.
Example of inanimate image that works nicely
II. The Photoface Editor – Suggestions & Tips
When you click on “Create Your Own” and initiate the Photoface process, there are 4 steps to go through:
1. Image upload & positioning
2. Anchor points placement
3. Masking the image
4. Finalizing your characters’ settings
Step 1 – Rotate & Scale
In the first step, your uploaded image is scaled and positioned automatically. Treat this initial positioning as a “first approximation”. It is up to you to fine-tune it.
Rotation: be sure to position the face with the eyes horizontally aligned. In some photos the mouth and the “eye line” are not parallel – your priority should be to align the eyes. The mouth can be fine-tuned later.
Scale: As you position the photo in the frame, you may want to zoom out a bit, to allow more of the body into the frame. This positioning determines your final character’s visible extent. If the body is cut too short it may limit later positioning on your web page.
Make sure the eyes are horizontally aligned, adjust mouth later
Step 2 – Anchor Point Placement
In the second step we auto-position the anchor points for you based on facial image analysis. This should also be considered an “initial approximation” for you to review and edit.
Follow the positioning in the sample picture & adjust the points carefully as needed. The auto-placement frequently requires manual fine-tuning.
Sometimes initial point positioning is off – always check and adjust
Step 3 – Masking the Image
We recommend going with the deafult settings in this step, unless you have good reason not to. Automasking works well in most cases, and produces better results.
Step 4 – Final Settings
In the final step, detailed tools are provided to help you get the mouth just right. Try them out repeatedly, until you get a good feel for how each setting affects the result.
For example, you may need to extend the mouth slit – not too wide, and not too narrow, to enable the mouth to open naturally. You may need to position the mouth slit off center & play around with its curvature and vertical location. Faces are never symmetrical.
Check out the slider for teeth whiteness. Avoid teeth that are too bright – a good rule of thumb is to match the whites of the eyes in the photo.
The “Camera Angle” setting can be very helpful. Play around with it to get a good feel for its effect.
Many times a face appears perfectly natural in the photo, and it is not immediately obvious that the photo was taken from slightly below the facial plain. This may cause the upper part of the head to stretch unnaturally as the head moves & can be a particular problem with naturally elongated faces.
Adjusting the camera angle can compensate & resolve.
Comments? Questions? Send us a note to: support@sitepal.com
We’ve just completed our first 3rd Party Text-To-Speech integration – with Google Cloud TTS leading the way. This integration adds 284 Google voices to SitePal.
Before proceeding we’d like to make clear:
Using the new voices requires integrating your Google Cloud account with your SitePal account. This is done by adding your Google API key to your SitePal account’s “Connect” page. This is not difficult and requires no coding.
Thus the cost of generating the Google voices (if any) is borne by you, and not included in the SitePal subscription cost. Please bear in mind.
3rd Party TTS voice integration requires the SitePal Platinum Plan.
The integration covers Google’s Standard, Wavenet & Neural voices. For a complete list of the added voices, see our API Reference – see Engine 11 in Appendix B.
Note: some newly added Google Voices might not yet be available in SitePal, we’ll be adding those periodically.
Google voices can now be used in SitePal just like any of our built-in voices. The new voices can be used either Statically or Dynamically.
Statically – created using the UI in your SitePal account, and then assigned to your Scenes.
Dynamically – real time via the SitePal API. Response times and performance are excellent.
Tips:
If the audios that your avatar will speak are known in advance, it would be best to manually create, name & save the audios in your SitePal account, and then use the “sayAudio” API call to invoke speech dynamically (as opposed to using “sayText”). Doing so means that we will not access the Google API at runtime, thereby saving usage costs.
Separately – it’s worth noting that SitePal caches dynamic TTS and if an audio is frequently used it will likely be delivered from the cache. This optimization is built-in – you don’t need to do anything to turn it on.
We’re working on integrating additional 3rd Party TTS providers, with AWS Poly, MS-Azure and Eleven Labs next on our list.
Last but not least! – we’ve recently introduced great new Robotic characters, available now to all SitePal plans. Check out our entire collection here: www.sitepal.com/collection
New Robot avatars. Each of them has several alternates.
With recent AI technology advances, we’re seeing the long sought after vision of a “Virtual Doctor” come closer to reality!
While a virtual healthcare professional is a prominent (and much needed) use case – this has enormous potential which is not limited to healthcare. The gap between available services and the public’s ability to take advantage of them is growing in every industry. With available options growing or changing by the day, it is likely that your customers cannot effectively understand or use your products without talking to someone.
We recently implemented integration with ChatGPT, which opens the door to any “Virtual Expert” type application. Industries from Insurance to Finance to Travel to Government Services and everything in between, can and will take advantage of AI Agents to make their products more accessible to the public.
How often have you heard the phrases: “If I could only talk to someone who understands all this!” – well, now the technology exists to truly make that happen.
To learn more about creating an AI Agent with SitePal and integrating with ChatGPT (or any 3rd party AI solution) see our recent blog entry – www.sitepal.com/blog?p=239.
So, with all that in mind we’ve just added several new healthcare professionals to our lineup. To review our full lineup of characters see: www.sitepal.com/collection, or create your own character from a photo using Photoface. If you need a custom character for your application – let us know.
We’re constantly working on new SitePal features & capabilities (and great new characters too!). Stay tuned for more updates…
ps. Why type your questions when you could speak them? Voice based interaction would be so much better! An idea whose time has come? We think so. Check out our latest AI Speech demo here – www.aispeechdemo.com
Hello SitePal Fans!
We are delighted to let you know that SitePal is now pre-integrated with ChatGPT.
What does this mean, and how can you take advantage of it?
SitePal has long supported building intelligent agents using its built-in AIML based capability. This means that you can create a knowledge base (a ‘Bot’) which contains relevant information for your visitors, and use a SitePal character as an intelligent agent to communicate with your website visitors. Your website visitors can learn about your products and services by conversing with your SitePal character.
We’ve now enhanced this capability with connectivity to 3rd party bots. The first such integration was with Pandorabots several weeks ago, now followed by OpenAI’s ChatGPT.
The following technical example allows you to select which Bot to interact with. Select ChatGPT from the menu & try it out by asking questions.
So what do you need to do to create a useful ChatGPT implementation for your website?
In broad strokes the effort would consist of 2 parts:
1. Open an account with OpenAI and teach your ChatGPT bot about your products (or anything else you want it to know)
2. Setup a SitePal character on your website to answer users questions by interacting with your ChatGPT bot.
The first part can be implemented using tools provided by OpenAI and other companies. This is outside the scope of our explanation here.
The second part is where our Pre-Integration comes into play. To publish your SitePal character as a speaking ChatGPT driven inteligent agent, you need to add your OpenAI API key to your SitePal account. Then use our sayAI API call (yes, ONE call !) to have your character speak the responses to user questions.
We did the heavy lifting – so you don’t need to! It’s that easy.
Please see detailed step-by-step instruction in the following technical note – and start taking advantage of intelligent agent technology today!
Additional 3rd party AI vendors will be integrated with SitePal in the coming months. If you have a specific one in mind, drop us a note and let us know. We will do our best to prioritize.
ps. Why type your questions when you can speak them? Talking to your bot would be so much better! An idea whose time has come? We think so. Check out our new AI Speech Demo here – www.aispeechdemo.com
We’ve received many inquiries about using SitePal with React Native, and we’ve worked hard to make that possible.
React Native makes building mobile apps much simpler as many developers are coming to realize. This is mainly because it allows web developers to capitalize on their existing expertise, but also because it is a cross platform tool that enables you to build native apps for both iOS and Android.
ps. If you are just starting out with SitePal, or it’s been a while since you used SitePal (so much has changed…) we encourage you to check out our video tutorials, available on our support page here – www.sitepal.com/support
Customers sometimes ask us whether SitePal characters’ lip animation can be improved.
Well, if you are using human recorded audio then the answer is yes! We recently launched a very useful update we’d like to make you aware of.
Note:
This improvement applies only to 3D characters & works only for English for now
For TTS – the improvement is automatically applied – see explanation below
When you upload (or record) an audio, you will now see a “transcript” section where you can add… yes, a text transcript of the audio.
The text transcript is used to improve our automated lip-sync analysis and produce better results.
How much better?
We put together two SitePal Scenes for you to compare. “Before” and “After” if you will. Unlike the Pepsi challenge – you’ll need to keep your eyes open! Take a look:
Uploaded without transcript: www.sitepal.com/geturl/?ss=2756599&sl=0&acc=7029566
Uploaded with transcript: www.sitepal.com/geturl/?ss=2756598&sl=0&acc=7029566
As you can see the improvement is noticeable. So when adding audios to your account do take the time to add a text transcript to each audio, it is well worth the effort. To apply this improvement to existing (recorded or uploaded) audios, you would need to re-upload your audios and enter a text transcript for each.
The other (unrelated) advantage of adding a text transcript to your audio is that it enables subtitle display. If subtitles are relevant to your implementation – we suggest reading the technical note “Adding Subtitles to your Scene”, on our support page.
Note:
for TTS (both static and dynamic) adding a transcript is usually unnecessary because the text used to generate the audio is automatically applied to enhance the Lip-Sync as well. So there’s nothing you need to do. But in some cases you may still benefit from adding transcript to your TTS text input. That might be the case if your TTS text is heavily phonetic, or contains SSML tags. A good way to think about it is: TTS text is meant to be machine readable. The transcript is meant to be human readable (whether used for Lip-Sync or subtitles or both). To the extent the TTS text is modified to enhance the former, you might benefit from the latter.
Oh, before we forget – do check out our new AI Demo – if you haven’t done so already! See – www.aispeechdemo.com
We’re actively working on additional features & capabilities (and new characters!) to be added soon… Stay tuned for more information.
( tip: click on the “die” button to select other SitePal characters )
The point of this demo is to showcase 2-way speech interface, coupled with SitePal integration. The specific AI bot integrated here is not the point – as SitePal can be used as the animated face for any AI chat bot.
To implement the speech recognition part we’re using the Web Speech API. The Web Speech API is an evolving web standard which is now built-in to most browsers* and can be used for speech recognition.
A technical example is also available on our support pages – where you can more easily peruse the source code and see how it is put together. See at:
As always – our examples are meant to be copied and used – so check out the source code of the page and feel free to use it with your SitePal account.
To learn more about implementing SitePal for your AI bot, send us a note to support@sitepal.com & ask for guidelines.
*Note: the Web Speech API is universally supported on all platforms and browsers – including mobile -with the exception of the Firefox browser. The demo (and example) automatically reverts to text input when Firefox is used.
Oh, and last but not least – we’ve just added new animal characters, now available to all SitePal customers. Check out our entire collection here – https://sitepal.com/collection
As always, we’re actively working on additional features & capabilities (and new characters!) to be added soon… Stay tuned for more information.
