This is the second post in the series. I need to start with a basic working example. And that means having a client that can talk to a server and a server that can talk back to a client. My objectives were as follows:
- Setup a client and a multithreaded server
- Each thread handles a different client on the server
- Clients communicate the user's intentions and the server sends back the actual position.
The Plan
In a multiplayer game the heart of it comes down to the network model. i.e how you transmit and receive information for multiple clients to play a game acceptably well.1
The first question you have to ask yourself about the network model is the topology. P2P or Client-Server. P2P is very possible but it does not scale well. After say 10 clients, overhead gets too much. Furthermore, how do you establish trust in a P2P setting ? Too many unknowns in this direction so I went with the client-server model.
This is what I had sketched out before coding to see where I needed to go.
To break down the process
- Client sends a
{JOIN XYZ GAME}request to the server - Server makes a thread for the client and checks if GAME
XYZis available on the server - Server sends back Client's initial position in the game
- Client render's the "Bob" in that position.
- Client initiates the event loop locally and listens for key presses
- Client sends the user's intentions to the server
- Server calculates the client's position (and Server broadcasts the client's position to others)
- Client renders the Bob based on the server's message.
Setup
The essential structure of the code was as follows
├── CMakeLists.txt
├── README.md
└── src
├── client
├── client_main.cpp
├── server
└── server_main.cpp
The client directory held all the client code that was run in the client_main.cpp. It generated the mazefighter-client executable. The same setup for the server generated the mazefighter-server executable. CMake was used as the build tool.
Sockets Refresher
To get the client and the server talking, we needed to setup sockets over the network for them to communicate over. My idea was the following
- The server accepts a client via TCP
- The server then starts a new thread with an ephemeral UDP socket and uses that to communicate with the client
- The client opens a UDP socket too and then communicates with the server's client-specific UDP socket.
Client side
The client setup that I arrived at was as follows
class GameClient {
public:
GameClient();
~GameClient();
void start();
void terminate();
private:
std::atomic<bool> m_running;
int m_tcp_socket;
int m_udp_socket;
std::thread m_conn_thread;
std::thread m_udp_recv_thread;
void make_connections();
void send_udp_packets(std::string msg);
};
The m_tcp_socket would connect to the server and transmit any preliminary data and then m_udp_socket would then transmit data about its directions. The connection thread would be processed seperately. (n)
// Create TCP socket
m_tcp_socket = socket(AF_INET, SOCK_STREAM, 0);
// ...
// Create UDP socket
m_udp_socket = socket(AF_INET, SOCK_DGRAM, 0);
// ...
// Bind UDP socket to receive packets
struct sockaddr_in udp_addr;
udp_addr.sin_family = AF_INET;
udp_addr.sin_port = 50001;
udp_addr.sin_addr.s_addr = INADDR_ANY; // Late binding
if (bind(m_udp_socket, (struct sockaddr*)&udp_addr, sizeof(udp_addr)) < 0) {
// ...
}
std::cout << "Client address: " << inet_ntoa(udp_addr.sin_addr) << ":" << ntohs(udp_addr.sin_port) << std::endl;
The inet_ntoa function prints the IP address of the client and the ntohs converts from the little-endian network representation to the system representation for the port number. This prints the local UDP address. The tcp socket will be used to connect and so there is no need to bind.
Why late bind ? Why not have a fixed UDP port number ?
- Because I will be testing multiple client on the same machine with the sever on the same machine (all local for now)
- Hence these two seperate clients will need to have different UDP port numbers to communicate.
Now that we have a UDP socket and a TCP socket on the client, we need to connect to the server. This is done with.
void GameClient::make_connections() {
struct sockaddr_in server_addr;
server_addr.sin_family = AF_INET;
server_addr.sin_port = htons(50000);
// For now the server is at this address.
server_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
if (connect(m_tcp_socket, (struct sockaddr*)&server_addr, sizeof(server_addr)) < 0) {
std::cerr << "Failed to connect to server" << std::endl;
m_running = false;
return;
}
std::cout << "Connected to server via TCP" << std::endl;
// ...
The problem is now that the server and the client do not know how to exchange information via UDP. Remeber UDP is spray and pray. They just route it via sendto. But I can't send it to the server's address specified above since that's the TCP server socket not the ephemeral UDP socket.
int sendto(socket, packet, sizeof(packet), 0,(struct sockaddr *)&server_addr, addr_len);
int recvfrom(socket, packet, sizeof(packet), 0,(struct sockaddr *)&server_addr, &addr_len);
How do I know what the server's UDP port is ?
- With the TCP connection, we can exchange information about the sockets and port number and the server and client can accordingly
- The server needs to know hte client's UDP port number
- The client needs to know the server's ephemeral UDP port number.
This is what I worked on next
// Send the client's intentions (Game to join and its UDP port)
char buffer[1024];
memset(buffer, 0, 1024);
snprintf(buffer, 1024, "%u,%d", m_game_id, udp_port);
if (send(m_tcp_socket, buffer, sizeof(buffer), 0) < 0) {
perror("Failed to send the game state");
exit(EXIT_FAILURE);
}
// Receive the new port from the server
memset(buffer, 0, 1024);
if (recv(m_tcp_socket, buffer, 1024, 0) < 0) {
perror("Failed to recieve port from client");
exit(EXIT_FAILURE);
}
int server_udp_port;
// Parse the UDP port and the Player ID
if (sscanf(buffer, "%d,%u", &server_udp_port, &m_player_id) != 3) {
std::cerr << "Failed to parse buffer" << std::endl;
}
std::cout << "The server sent the following port " << server_udp_port
<< " and the following player id " << m_player_id << std::endl;
// Set the new port for the server
server_addr.sin_port = htons(server_udp_port);
return;
}
Now that we have the server address for the data transmission, we can proceed.
Server side
This was quite reciprocal for the server. The server would accept communications via its TCP socket. Then it would send it through to a handle client function that is responsible just for communicating with the client.
This was the accept-loop
while (m_running) {
struct sockaddr_in client_addr;
socklen_t client_len = sizeof(client_addr);
// Accept the client connection
int client_socket =
accept(m_tcp_socket, (struct sockaddr *)&client_addr, &client_len);
// ...
// Receive the client's information via the TCP socket
uint32_t game_id = 0;
int client_udp_port = 0;
if (recv(client_socket, buffer, 1024, 0) > 0) {
// game_id and client_udp_port get populated here.
} else {
perror("Failed to recieve the client's game id");
}
// ...
// Set the client's
std::thread client_connection = std::thread(&GameServer::handle_client, this, client_socket, client_addr, game_id);
client_connection.detach();
}
SFML Rendering
Now that we have sockets that ping each other back and forth, how we do render stuff ?
Introducing SFML (Simple Fast Multimedia Library)
Its a very fast and easy-to-use C++ library that is used to build games. Essentially it provides all the cross-platform graphic, sprite, sound support so that you can focus on the game. It was as easy as
# Find SFML packages
find_package(SFML 2.5 COMPONENTS graphics window system REQUIRED)
#...
# Link SFML libraries for both executables
target_link_libraries(mazefighter-server PRIVATE
sfml-graphics
sfml-window
sfml-system
)
target_link_libraries(mazefighter-client PRIVATE
sfml-graphics
sfml-window
sfml-system
)
Now I could bind the SFML libraries to the exectuble and develop with it. So I started with the very simple "Bob". A neat little 50x50 rectangle.
// Create the player rectangle
m_playerRect.setSize(sf::Vector2f(50, 50)); // 50x50 square
m_playerRect.setFillColor(sf::Color::Blue);
m_playerRect.setPosition(m_playerPos);
All in all, it got me here
An Authoritative Server
Now this is a multiplayer competitive game. How do we ensure that client's don't cheat. i.e how much can we trust the client ? Seeing that this is a competitive game, the answer is not much. So what does that mean ? It means the only source of truth about the player's location and actions is the server.
- The client sends what it wants to do to the server
- The server parses these actions and then sends the client its in-game location.
So we need to wait for events on the client-side. This is done with SFML Keyboard events
// Handle continuous keyboard input while the window is in Focus.
if (m_window.hasFocus()) {
if (sf::Keyboard::isKeyPressed(sf::Keyboard::W)) {
if (m_playerPos.y > 0) {
// m_playerPos.y -= moveSpeed;
send_message("up");
update_game_state();
}
}
if (sf::Keyboard::isKeyPressed(sf::Keyboard::A)) {
if (m_playerPos.x > 0) {
// m_playerPos.x -= moveSpeed;
send_message("left");
update_game_state();
}
}
if (sf::Keyboard::isKeyPressed(sf::Keyboard::S)) {
if (m_playerPos.y <= (m_window.getSize().y - rectSize.y)) {
// m_playerPos.y += moveSpeed;
send_message("down");
update_game_state();
}
}
if (sf::Keyboard::isKeyPressed(sf::Keyboard::D)) {
if (m_playerPos.x <= (m_window.getSize().x - rectSize.x)) {
// m_playerPos.x += moveSpeed;
send_message("right");
update_game_state();
}
}
// Update rectangle position
m_playerRect.setPosition(m_playerPos);
}
Upon every key press, the client will transmit its direction to the server. Then it will wait for the server to send back its position. On localhost this was almost instantaneous. The server was quite similar to the client
while (1) {
int bytes_received =
recvfrom(udp_socket, buffer, 1024, 0,
(struct sockaddr *)&client_addr, &addr_len);
if (bytes_received > 0) {
std::lock_guard<std::mutex> lock(m_print_mutex);
// std::cout << "[P #" << new_player_id << "] " <<
// std::string(buffer, strlen(buffer)) << std::endl;
process_action(new_player_id, game_id, buffer);
broadcast_state(game_id);
} else if (bytes_received < 0) {
perror("Failed to receive bytes");
}
memset(buffer, 0, 1024);
}
It would receive the bytes from the client. Process this action (i.e interpret it) and then transmit the state of play to ALL THE CLIENTS CONNECTED. This is also the reason for the lock. We cannot have a client join while the server is broadcasting. This completes all the steps in our plan.
Other players
Now all that's left is actually having other players. In order to accomodate other players, the client needs to support their state updates.
// other players
std::map<uint32_t, std::tuple<sf::RectangleShape, sf::Vector2f>> others;
The client would broadcast an entire game state. This was
\{GAMEID}
\{PLAYER1_ID}(PLAYER1_x:PLAYER1_y)
\{PLAYER2_ID}(PLAYER2_x:PLAYER2_y)
which over the wire looked something like this. (Captured via Wireshark).
0000 02 00 00 00 45 00 04 1c cb 2b 00 00 40 11 00 00 ....E....+..@...
0010 7f 00 00 01 7f 00 00 01 c3 52 c9 15 04 08 02 1c .........R......
0020 31 33 30 35 37 38 31 31 33 38 0a 28 31 32 33 33 1305781138.(1233
0030 33 32 36 34 39 32 29 33 30 30 2e 30 30 30 30 30 326492)300.00000
0040 30 3a 33 31 31 2e 30 30 30 30 30 30 0a 28 33 34 0:311.000000.(34
0050 39 35 34 38 38 32 33 30 29 34 31 30 2e 30 30 30 95488230)410.000
0060 30 30 30 3a 33 33 35 2e 30 30 30 30 30 30 0a 00 000:335.000000..
0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
// ...
The serialization was inefficient of course (cause I have ot process strings) but that was a problem for a later time. The client would read this, process it player by player.
- Case 1: If it sees a player Id that is not its own => its another player. If this player does not exist in its map => its a new player. Add it to the map.
- Case 2: If it sees a player Id that is not its own => its another player. If this player does exist in its map => its a state update. Update that player's location.
- Case 3: If it sees its own player Id => its a state update. Update its state and render.
That looked something like this.
Why is there a delay in the sending and receiving of updates across clients?
- The problem above was that I waited for updates and sent updates on the same thread.
- This meant that the client wouldn't receive updates until it sent some data
- This was an obvious issue that I had to fix but the basics were in place.
Lessons Learned
-
Parkinson's Law is real. So lets use it. I gave myself a full week to go from zero-to-hero in this game and this setup. If I had given myself a month, it would have yielded the same result. Given this, it can be a superpower to fixate on a deadline. Of course, not too unrealistically. Be reasonable. I've also learnt from 16 types of useful predictions — LessWrong. This is something I intend to do for future iterations of this project. Predict how long it will take. Predict what will be necessary. When you're done, review and see how well you performed. Keep up this loop. Initially it will be hard. But soon with enough forced practice it will become subconscious. You build models of the world and of time and of your capabilities.
-
I love simple devlogs or tutorials. There's nothing more satisfying to me as an engineer than someone who does something and then talks about it in technical detail. Not a course. Not a textbook. But someone who has actually built the thingy and documents its intricacies. The flaws. The setbacks, etc. And most of all - keeps it simple. No fancy page. No ads. No graphics. Just text, code and some illustrations. Here's the ones I used: High Hat (s/o to my mate Devin Leamy for introducing me to this), GameCodeSchool, Gabriel Gambetta and Multiplayer FPS. There's plenty
Footnotes
-
acceptably well can mean a lot of things. In my case it was < 30ms latency and no jitter / crashes client-side. ↩