Rails7英语学习项目从零开始-2-语音聊天原创邦

Rails7英语学习项目从零开始-2-语音聊天

接上: Rails7英语学习项目从零开始

计划做一个在线的英语角，新增在线聊天功能。
涉及到的ActionCable和WebRTC相关的功能。WebRTC相对来说复杂一点，需要掌握多一点的前端知识。

大致流程：张三、李四、王五都进入到room1里面，每个人获取到其它人的media，并展示在自己的界面上。

MVC
1 新建model ： room(语言，话题，最大人数，创建人）
2.新建View页面，room列表+ 进入room里面
3.Room Controllers（index，show，new，create）进入show就是进入房间，用户订阅@room,

ActionCable-Channel的后端代码
2.1 张三进入room需要广播给room里面的其他成员。这里需要一个room_channel, 通知其他成员增加张三的media。
#app/channels/room_channel.rb

classs RoomChannel < ApplicationCable::Channel
  # 用户订阅room频道，调用的方法。
  def subscribed
    @room = find_room
    stream_for @room  # 订阅这个model的广播，获取这个room的GlobalID。
    broadcast_to @room, { type: 'ping', from: params[:user_id]} #通过broadcast_to @room广播到这个@room频道
  end

  def unsbscribed
    Turbo::StreamChannel.boardcast_remove_to(find_room, target: "medium_#{current_user.id}")
  end

  # 发送相关数据给新进入的订阅者
  def greet(data)
    Turbo::StreamChannel.boardcast_append_to(data['to'], target: 'media',partial: 'media/medium', locals: {client_id: data['from']})
  end

private
  def find_room
    Room.find(params[:id])
  end
end

2.2 创建一个room里面所有用户之间建立对话传递信号的频道: SignalingChannel, 用户间建立p2p连接的webRTC协商时候需要的SDP descriptions和ICE candidates通过signal来exchange数据。

#app/channels/singnaling_channel.rb
class SignalingChannel < ApplicationCable::Channel
  def subscribed
    stream_for Room.find_by(id: params[:id])
  end
  # 给room发信号
  def signal(data)
    broadcast_to(Room.find_by(id: params[id], data)
  end
end

2.3 用户订阅Room, application_cable ,用户连接到ActionCable需要一个用户标示。

#app/channels/application_cable/connection.rb
module ApplicationCable
  identified_by current_user
  def connect
    self.current_user = find_user
  end

  private
  def find_user
    User.find_by(id: cookies.encrypted[:user_id])
  end
end

JS前端
用户进入房间，连接到cable， connect()
show页面上传递user_id, room_id 传递给stimulus的 room_controllers.js
点击进入房间，js

#app/javascript/controllers/room_controller.js
import { Controller } from '@hotwired/stimulus'
...
...
export default class RoomController extends Controller {
  connect() {
    this.users = {}
    this.subscription = new RoomSubscription({delegate: this, id: this.idValue,userId: this.user.id})  #订阅room
    this.signaller = new Signaller({delegate: this, id: this.idValue, userId: this.user.id})  #订阅信号频道，为建立webRTC negotiation提供信息服务。
    this.user.on('iceConnection:checking', ({ detail: { otherUser } }) => { this.startStreamingTo(otherUser)}) # 正在检查的时候，提交自己的stream。
  }
  async enter (){
    try{
      // 获取本地的音视频，添加到页面的Dom里面，
      // 订阅room，和room里其他用户建立链接。

    } 
    catch (error) {console.error(error)
    }
  }
}

iceConnections:checking

The ICE agent has been given one or more remote candidates and is checking pairs of local and remote candidates against one another to try to find a compatible match, but has not yet found a pair which will allow the peer connection to be made. It is possible that gathering of candidates is also still underway.
ICE代理已被给予一个或多个远程候选，并正在相互检查本地和远程候选对，以尝试找到兼容的匹配，但尚未找到允许建立对等连接的对。候选人的聚集也可能仍在进行中。

#app/views/rooms/show.html.slim
= turbo_stream_form @room
= turbo_stream_from @client.id

turbo_stream_from的作用
----------------------------------

Turbo::StreamChannel The streams channel delivers all the turbo-stream actions created (primarily) through Turbo::Broadcastable.

SDP

Stream_for 和 Stream_from

https://guides.rubyonrails.org/action_cable_overview.html#streams

如果你有一个stream和一个model相关，广播这个name，用stream_for去订阅一个广播。
If you have a stream that is related to a model, then the broadcasting name can be generated from the channel and model. For example, the following code uses stream_for to subscribe to a broadcasting like comments:Z2lkOi8vVGVzdEFwcC9Qb3N0LzE, where Z2lkOi8vVGVzdEFwcC9Qb3N0LzE is the GlobalID of the Post model.
你可以通过下面的代码广播到这个频道。

class CommentsChannel < ApplicationCable::Channel
  def subscribed
    post = Post.find(params[:id])
    stream_for post
  end
end

You can then broadcast to this channel by calling broadcast_to:

CommentsChannel.broadcast_to(@post, @comment)

stream_for(model, callback = nil, coder: nil, &block) public

Start streaming the pub/sub queue for the model in this channel. Optionally, you can pass a callback that’ll be used instead of the default of just transmitting the updates straight to the subscriber.
this channel里开启 the model 发布/订阅的streaming, 你可以传一个callback，而不只是传更新的内容给订阅者。
Streams provide the mechanism by which channels route published content (broadcasts) to their subscribers. For example, the following code uses stream_from to subscribe to the broadcasting named chat_Best Room when the value of the :room parameter is "Best Room":

pub/sub 发布/订阅模式
The publish/subscribe (pub/sub) pattern is a simple but powerful architectural design pattern which can benefit your messaging application design.
帮助你消息传递的设计模式

还有个声明订阅广播的方法是
Stream_from
Stream提供一个机制：频道路由公布内容给他的订阅者，通过stream_from去订阅这个广播，名字为""
Streams provide the mechanism by which channels route published content (broadcasts) to their subscribers. For example, the following code uses stream_from to subscribe to the broadcasting named chat_Best Room when the value of the :room parameter is "Best Room":

# app/channels/chat_channel.rb
class ChatChannel < ApplicationCable::Channel
  def subscribed
    stream_from "chat_#{params[:room]}"
  end
end

通过下面的代码去发送广播

ActionCable.server.broadcast("chat_Best Room", { body: "This Room is Best Room." })

××××××××××××××××××××××××××××××××××××××××××××××
SDP (Session Description Protocol)

會話描述協議（Session Description Protocol 或簡寫 SDP）描述的是流媒體的初始化參數。此協議由 IETF 發表為 RFC 2327。

SDP 格式

v=0
o=mhandley 2890844526 2890842807 IN IP4 126.16.64.4
s=SDP Seminar
i=A Seminar on the session description protocol
u=http://www.cs.ucl.ac.uk/staff/M.Handley/sdp.03.ps
e=mjh@isi.edu (Mark Handley)
c=IN IP4 224.2.17.12/127
t=2873397496 2873404696
a=recvonly
m=audio 49170 RTP/AVP 0
m=video 51372 RTP/AVP 31
m=application 32416 udp wb
a=orient:portrait

v=協議版本
o=發起者的 Session、Session ID 及 Session 版本
s=Session 名字
i=Session 資訊
u=有關會議資訊的 url
e=Email
p=手機號碼
c=連線資訊
t = Session 活動時間
m = 媒體資訊 ((media) (port) (transport) (fmt list))
a = 媒體屬性

Ice Candidate

Ice Candidate 描述 WebRTC 能與 遠程設備通訊所需的協議和路由，啟動 WebRTC P2P 後，通常會在連接的每一端提供多個 IceCandidate，直到絕定最佳線路達成為止．

{
  "sdpMLineIndex": 0,
  "sdpMid": "",
  "candidate": "a=candidate:2999745851 1 udp 2113937151 192.168.56.1 51411 typ host generation 0"
}

WebRTC Flow

雙方 Peer 先連上 Signaling Server
PeerA 取得自身 SDP 並呼叫 setLocalDescription
PeerA 將 SDP 傳給 Signaling Server
Signaling Server 將 PeerA 的 SDP 送給 PeerB
PeerB 呼叫 setRemoteDescription 將 PeerA 的 SDP 寫入
PeerB 取得自身 SDP 並呼叫 setLocalDescription
PeerB 將 SDP 傳給 Signaling Server
Signaling Server 將 PeerB 的 SDP 送給 PeerA
PeerA 呼叫 setRemoteDescription 將 PeerB 的 SDP 寫入
PeerA 向 Stun server 詢問 public IP
Stun server 回應 public IP
PeerA 向 TURN server 詢問 relay 資訊 (relay ip/port)
TURN server 回應 relay 資訊
PeerA 將 Ice candidates 傳給 Signaling Server
Signaling Server 將 PeerA 的 Ice candidates 送給 PeerB
PeerB 呼叫 setRemoteIceCandidate 將 PeerA 的 Ice candidates 寫入
PeerB 向 Stun server 詢問 public IP
Stun server 回應 public IP
PeerB 向 TURN server 詢問 relay 資訊 (relay ip/port)
TURN server 回應 relay 資訊
PeerB 將 Ice candidates 傳給 Signaling Server
Signaling Server 將 PeerB 的 Ice candidates 送給 PeerA
PeerA 呼叫 setRemoteIceCandidate 將 PeerB 的 Ice candidates 寫入
P2P 通道建立完成

AWS KVS (Amazon Kinesis Video Streams)

Amazon Kinesis Video Streams 以全受管功能提供符合標準的 WebRTC 實作。您可以使用 Amazon Kinesis Video Streams and WebRTC 安全地即時串流媒體，或在任何攝影機 IoT 裝置與符合 WebRTC 標準的行動或 Web 播放器之間，執行雙向音訊或視訊互動。因為是全受管功能，您不需要建置、執行或擴展任何與 WebRTC 相關的雲端基礎設施，例如訊號或媒體轉送伺服器，即可在應用程式和裝置之間安全地串流媒體。

簡單來說 KVS 就是幫你把 STUN, TURN, Signaling Server 加密權限驗證等等都實作了，WebRTC 的部分跟 KVS 是完全獨立的，你也可以選擇自己架設 STUN, TURN, Signaling Server 搭配 Google WebRTC 也能成功串流．

KVS 的 Signaling server 是用 WebSocket 去實作的

注意: WebSocket 與 Socket.IO 是不是一樣的，Socket.IO 是根據 Websocket 協議去實作，Socket.IO 有自己的通訊格式，請不要拿 Socket.IO 套件去串接 KVS，會失敗，有興趣可以參考這篇【筆記】Socket，Websocket，Socket.io 的差異

阅读量: 885
发布于: 2023-03-07
修改于: 2023-03-10