Webrtc和rails



The Stimulus room controller handles Enter button click. It gets the user's local audio and video and feeds them into the local video element. It also starts Action Cable subscriptions specific to the current room: one for communicating WebRTC messages: the Signaller; and one for clients to ping others: the RoomSubscription. Once connected, the room channel broadcasts the presence of this new client. Each connected client then "greets" this newcomer by calling the greet action on the room channel and specifying to and from fields.

Stimulus room 控制器处理 enter 按钮点击。
它得到用户的local audio和video,并加入到本地的video元素element中。
它也启动对当前room  的 Action Cable订阅: 一个WebRTC消息:Signaller;  and one for clients 去ping其它的clients:RoomSubscription.
一旦连接上了,这个room channel 广播这个新客户的存在。
每一个连接client "greets" 新来的,通过在room频道上调用greet动作,指定去和来的field。

The greet action broadcasts specifically to the newcomer, a Turbo Stream update which appends a media element representing the remote client. It's important we do this first so that it's ready for incoming streams. The medium controller messages the room controller to notify that it has been added. This begins the WebRTC negotiation to form a connection between the two clients.

这个greet动作广播给新来的。一个Turbo Stream更新会追加一个media元素表示远程的client。 这个表示为新进来的streams做好了准备。
Medium控制器通知 room 控制器去 通知 已经增加了。
开始WebRTC协商去在两个客户端中形成一个连接。

The WebRTC negotiation is quite complex, and even though things are required to happen in a particular order, responses are triggered asynchronously, making it tricky to get right. This is where the WebRTC Perfect Negotiation pattern comes in. We won't go into it too much here, as it's covered well elsewhere; but for the purpose of this description, (deep breath), Session Description Protocol (SDP) descriptions and Interactive Connectivity Establishment (ICE) candidates are exchanged over the Signaller. The negotiation is "perfect" as it ensures that the RTCPeerConnection is in the correct state for setting descriptions, and avoids collision errors by having one client be "polite" and other "impolite"—the polite one backs down in the case of a collision. Anyway, the code for this has been mostly lifted from the spec and handled by the Signaller/WebrtcNegotiation. When creating the negotition, the room controller listens for track events on the RTCPeerConnection, so it can start streaming from the remote client.

WebRTC协商非常复杂,即使事情需要按照特定的顺序发生,响应也会异步触发,因此很难正确处理。这就是WebRTC Perfect Negotiation模式的由来。我们在这里不做过多的讨论,因为其他地方也有介绍;但是为了本描述的目的,会话描述协议(SDP)描述和交互式连接建立(ICE)候选通过信令器交换。协商是“完美的”,
因为它确保RTCPeerConnection处于设置描述的正确状态,并通过让一个客户端“礼貌”和其他“不礼貌”来避免冲突错误-在发生冲突时,礼貌的客户端会后退。
无论如何,这方面的代码大部分已从规范中删除,并由Signaller/WebrtcNegotiation处理。创建neotion时,房间控制器侦听RTCPeerConnection上的跟踪事件,因此可以从远程客户端开始流式传输。

Once the negotiation has been created, the newcomer client notifies the other client by reciprocating the greeting i.e. calling greet on the room channel. This triggers the same process as above on the other client: appending a media element via a Turbo Stream update, and starting a WebRTC negotiation. This time though, instead of a greeting (which would be very polite, but awkward and endless!), the client starts streaming to the other client, by adding media tracks to the connection. This kicks off the complex exchange of SDP descriptions and ICE candidates, and once either is received by the newcomer, it can start the streaming process from its own stream to the other client. Now all clients are streaming to and from each other.

Finally, when a client disconnects from the room channel, a Turbo Stream update removes the media element from the screen. The media controller broadcasts its removal so the room controller can clean up.

To summarise the flow:

  1. A newcomer broadcasts its presence to others in the room
  2. The connected clients greet this newcomer letting them know their ID
  3. A Turbo Stream update creates a video element on the newcomer's screen for each greeting it receives
  4. The newcomer creates a WebRTC negotiation and sends a greeting back to each of the other clients
  5. Turbo Stream updates create video elements on each of the other clients' screens
  6. WebRTC negotiations are created by each of the other clients, and they start streaming to the newcomer
  7. Reacting to negotiation activity (SDP descriptions and ICE candidate exchanges), the newcomer starts streaming to other clients

一旦创建了协商,新客户就通过在房间频道上往复问候(即呼叫问候)来通知另一个客户。这将在另一个客户端上触发与上述相同的过程:通过TurboStream更新附加媒体元素,并启动WebRTC协商。不过,这一次,客户端不再打招呼(这会很礼貌,但很尴尬,而且没完没了!),而是通过在连接中添加媒体曲目,开始向另一个客户端发送流媒体。这启动了SDP描述和ICE候选的复杂交换,一旦新人接收到其中一个,它就可以开始从自己的流到另一个客户端的流传输过程。现在,所有客户端都在相互之间进行流式传输。

最后,当客户端从房间频道断开连接时,Turbo流更新会从屏幕上删除媒体元素。媒体控制器广播其删除,以便房间控制器可以进行清理。

总结流程:

新来者向房间里的其他人广播自己的存在
连接的客户向这位新人打招呼,让他们知道自己的ID
Turbo Stream更新会在新人的屏幕上为其收到的每个问候创建一个视频元素
新人创建WebRTC协商,并向其他每个客户端发送问候
Turbo流更新在其他客户端的每个屏幕上创建视频元素
WebRTC协商由其他每个客户创建,并开始向新客户发送流
对谈判活动(SDP描述和ICE候选人交流)作出反应后,新人开始向其他客户发送流

https://github.com/domchristie/webrtc-hotwire-rails
https://gornostal.github.io/talks/webrtc/#/1

阅读量: 775
发布于:
修改于: