添加链接
link管理
链接快照平台
  • 输入网页链接,自动生成快照
  • 标签化管理网页链接

资源分享 – Practical Shader Development – Vertex and Fragment Shaders for Game Developers 英文PDF下载

2023-08-10 作者你好,请问在Hand Tracking和Holistic Tracking这两个任务分别编译DLL库的时候,为什么Hand Tracking需要从Mediapipe的原始仓库中拷贝mediapipe\mediapipe\examples\desktop\hand_tracking文件夹下的BUILD文件到新建的目录mediapipe\mediapipe\examples\desktop\hand_tracking_test下。 而Holistic 不需要啊? funnywii 在 2024-04-30 15:34:35 评论 Mediapipe – 将Mediapipe HolisticTracking封装成动态链接库dll/so,实现在桌面应用中嵌入全身关节点识别、手势识别、抬手放手检测识别功能

Javascript – 接收websocket服务器发送base64编码的音频数据并进行播放

Javascript 发布于2023-08-31 阅读 852次 0 次评论 本文共4769个字,阅读需要12分钟。

1 Javascript 接收websocket服务器发送base64编码的音频数据并进行播放

在js中除了可以播放本地音频之外,在有些情况下音频数据是需要在远程服务器上生成的,在这种情况下需要接收服务器端的音频数据并在本地中使用js播放接收的音频数据。

假设现在音频数据是由一个websocket服务器生成的,并且将生成的音频数据进行base64编码,之后再发给用户。那么用户这边需要先将接收的base64字符串解码成音频二进制流,然后进行播放。

首先需要处理下 Web Audio API 在不同浏览器下的名称兼容问题,并初始化 AudioContext 实例,

var g_audio_context = null;
try {
    const AudioContext = window.AudioContext || window.webkitAudioContext;
    g_audio_context = new AudioContext();
} catch (e) {
    alert('Web Audio API is not supported in this browser. Chrome is strongly recommended!');

然后在websockt连接的onmessage函数中解码base64数据,解码base64编码的字符串为二进制数据主要使用了Base64Binary.js这个库,具体的链接在这,然后整个Base64Binary.js的代码如下

All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL DANIEL GUERRERO BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * Uses the new array typed in javascript to binary base64 encode/decode * at the moment just decodes a binary base64 encoded * into either an ArrayBuffer (decodeArrayBuffer) * or into an Uint8Array (decode) * References: * https://developer.mozilla.org/en/JavaScript_typed_arrays/ArrayBuffer * https://developer.mozilla.org/en/JavaScript_typed_arrays/Uint8Array var Base64Binary = { _keyStr : "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=", /* will return a Uint8Array type */ decodeArrayBuffer: function(input) { var bytes = (input.length/4) * 3; var ab = new ArrayBuffer(bytes); this.decode(input, ab); return ab; removePaddingChars: function(input){ var lkey = this._keyStr.indexOf(input.charAt(input.length - 1)); if(lkey == 64){ return input.substring(0,input.length - 1); return input; decode: function (input, arrayBuffer) { //get last chars to see if are valid input = this.removePaddingChars(input); input = this.removePaddingChars(input); var bytes = parseInt((input.length / 4) * 3, 10); var uarray; var chr1, chr2, chr3; var enc1, enc2, enc3, enc4; var i = 0; var j = 0; if (arrayBuffer) uarray = new Uint8Array(arrayBuffer); uarray = new Uint8Array(bytes); input = input.replace(/[^A-Za-z0-9\+\/\=]/g, ""); for (i=0; i<bytes; i+=3) { //get the 3 octects in 4 ascii chars enc1 = this._keyStr.indexOf(input.charAt(j++)); enc2 = this._keyStr.indexOf(input.charAt(j++)); enc3 = this._keyStr.indexOf(input.charAt(j++)); enc4 = this._keyStr.indexOf(input.charAt(j++)); chr1 = (enc1 << 2) | (enc2 >> 4); chr2 = ((enc2 & 15) << 4) | (enc3 >> 2); chr3 = ((enc3 & 3) << 6) | enc4; uarray[i] = chr1; if (enc3 != 64) uarray[i+1] = chr2; if (enc4 != 64) uarray[i+2] = chr3; return uarray; // modulize var factory = function (global) { global.Base64Binary = Base64Binary; (function(global, factory) { if (typeof module === "object" && typeof module.exports === "object") { // For CommonJS and CommonJS-like environments where a proper `window` // is present, execute the factory and get jQuery. // For environments that do not have a `window` with a `document` // (such as Node.js), expose a factory as module.exports. // This accentuates the need for the creation of a real `window`. // e.g. var jQuery = require("jquery")(window); // See ticket #14549 for more info. module.exports = global.document ? factory(global, true) : function(w) { return factory(w); } else { factory(global); // Pass this if window is not defined yet })(typeof window !== "undefined" ? window : this, factory);

之后通过AudioContext创建一个buffer source,使用解码后的音频二进制数据作为其原数据,然后进行播放,主要的示例代码如下

function PlayAudio(audio_base64_data)
    let array_buffer = Base64Binary.decodeArrayBuffer(audio_base64_data);
    g_audio_context.decodeAudioData(array_buffer, function (buffer) {
        console.log("decodeAudioData success: "+ audio_base64_data);
        PlaySound(buffer);
    },function (e){
        console.error("decodeAudioData failed: "+ audio_base64_data);
    function PlaySound(buffer)
        let audio_source = g_audio_context.createBufferSource();
        audio_source.buffer = buffer;
        audio_source.connect(g_audio_context.destination);
        audio_source.start(0);
m_Websocket.onmessage = function(event)
    if(event.data != null)
        PlayAudio(event.data);

使用上述代码就可以源源不断的解析Websocket服务器发来的音频数据进行播放,并且音频数据播放的非常连贯,没有出现声音断断续续的问题。

  • https://developer.mozilla.org/zh-CN/docs/Web/API/Web_Audio_API/Using_Web_Audio_API
  • http://johnwong.github.io/past/2013/09/11/play-music-using-web-audio.html
  • https://q.cnblogs.com/q/85723/
  • 本文作者:StubbornHuang

    版权声明:本文为站长原创文章,如果转载请注明原文链接!

    原文标题:Javascript – 接收websocket服务器发送base64编码的音频数据并进行播放

    原文链接:https://www.stubbornhuang.com/2773/

    发布于:2023年08月31日 14:55:03

    修改于:2023年08月31日 15:04:16

    声明:本站所有文章,如无特殊说明或标注,均为本站原创发布。任何个人或组织,在未征得本站同意时,禁止复制、盗用、采集、发布本站内容到任何网站、书籍等各类媒体平台。如若本站内容侵犯了原著者的合法权益,可联系我们进行处理。

    Javascript播放base64编码的音频数据
    当前分类随机文章推荐
    全站随机文章推荐