• Register

Gamieon is a privately owned entertainment software development company located in Tampa, Florida. Since October of 2004, we have aimed to provide quality video game software which emphasizes both intellectual and action-driven challenge to the gaming community. Gamieon depends on the talent of individuals working as a team to develop video games and video game engines with a focus on exceptional game play and surrealism.

Image RSS Feed Latest Screens
Dominoze Construction Kit
Blog RSS Feed Report abuse Latest Blog: How I synchronized a simple rigidbody across network players with UE4

0 comments by Gamieon on Dec 17th, 2014


The Issue

I'm developing an online soccer game for UE4 which you can get from Github.com for now. During game play, the soccer ball can be in one of two states: Freely moving; or in possession. When freely moving, the ball moves by physics simulation. When in possession, the ball is always in front of the possessing character.

I noticed during online testing that the ball position and velocity on the client instances would deviate from the server when freely moving. Thinking I was doing something wrong with replication, I went into the editor and tried every combination of replication flags to fix it to no avail. Some Googling on the matter did not reveal a solution.


The Solution

I resolved to just deal with the issue myself in the same way I did in my Unity projects using lessons from Developer.valvesoftware.com . The server would simulate ball physics, and the clients would constantly be fed the ball orientation from the server. The clients would use interpolation/extrapolation to smoothly move their instance of the ball to where the server says it should be.


Physics Simulation

On the server, the soccer ball physics are simulated and collision detection handled when the ball is not in possession. On clients I ensure the physics are never simulated and that collision detection is always off like so:

cpp code:
/** This occurs when play begins */

void AMagicBattleSoccerBall::BeginPlay()

{

Super::BeginPlay();


if (Role < ROLE_Authority)

{

// The server manages the game state; the soccer ball will be replicated to us.


// Physics however are not replicated. We will need to have the ball orientation

// replicated to us. We need to turn off physics simulation and collision detection.

UPrimitiveComponent *Root = Cast<uprimitivecomponent>(GetRootComponent());

Root->PutRigidBodyToSleep();

Root->SetSimulatePhysics(false);

Root->SetEnableGravity(false);

SetActorEnableCollision(false);

}

else

{

// Servers should add this soccer ball to the game mode cache.

// It will get replicated to clients for when they need to access

// the ball itself to get information such as who possesses it.

AMagicBattleSoccerGameState* GameState = GetGameState();

GameState->SoccerBall = this;

}

}



Replication

There are three ball properties that must be replicated:

  • Orientation - This is the position and rotation of the ball
  • Velocity - This is used for extrapolation. If the server is slow to replicate data, the client should be able to predict where the ball is going while waiting for more data to come in.
  • Timestamp - The other properties require a context in time for proper interpolation/extrapolation. Sure the ball was at XYZ...but when was it there?

I created a USTRUCT with these properties which I call FSmoothPhysicsState.

cpp code:
USTRUCT()

struct FSmoothPhysicsState

{

GENERATED_USTRUCT_BODY()


UPROPERTY()

uint64 timestamp;

UPROPERTY()

FVector pos;

UPROPERTY()

FVector vel;

UPROPERTY()

FRotator rot;


FSmoothPhysicsState()

{

timestamp = 0;

pos = FVector::ZeroVector;

vel = FVector::ZeroVector;

rot = FRotator::ZeroRotator;

}

};


The ball has a FSmoothPhysicsState which I define as such:

cpp code:
/** The soccer ball orientation on the server */

UPROPERTY(ReplicatedUsing = OnRep_ServerPhysicsState)

FSmoothPhysicsState ServerPhysicsState;

UFUNCTION()

void OnRep_ServerPhysicsState();


and each client tracks the last twenty states (defined as PROXY_STATE_ARRAY_SIZE) in the replication function:

cpp code:
void AMagicBattleSoccerBall::OnRep_ServerPhysicsState()

{

// If we get here, we are always the client. Here we store the physics state

// for physics state interpolation.


// Shift the buffer sideways, deleting state PROXY_STATE_ARRAY_SIZE

for (int i = PROXY_STATE_ARRAY_SIZE - 1; i >= 1; i--)

{

proxyStates[i]= proxyStates[i - 1];

}


// Record current state in slot 0

proxyStates[0] = ServerPhysicsState;


// Update used slot count, however never exceed the buffer size

// Slots aren't actually freed so this just makes sure the buffer is

// filled up and that uninitalized slots aren't used.

proxyStateCount = FMath::Min(proxyStateCount + 1, PROXY_STATE_ARRAY_SIZE);


// Check if states are in order

if (proxyStates[0].timestamp < proxyStates[1].timestamp)

{

UE_LOG(LogOnlineGame, Verbose, TEXT("Timestamp inconsistent: %d should be greater than %d"), proxyStates[0].timestamp, proxyStates[1].timestamp);

}

}



Timestamps

I previously wrote that the replicated properties require a context in time. Though clients gets server timestamps, a client's current time may not be exactly the same time as the server's. The clients need to know the server's time throughout the game for proper interpolation/extrapolation.

To accomplish this, the client does the following:

  1. Get the server's time
  2. Calculate the difference between the server's time and its own time, and stores it in memory
  3. Any time the client needs to know the server's time, the client will get its own time and add the value from step 2 to it.

I'll expand on these steps here:

Step 1

  1. The client gets its own system time and stores it in a variable we call "Tc"
  2. The client sends an RPC to the server requesting the server's system time
  3. The server gets the client RPC. The server then gets its own system time, and responds to the client with that value.
  4. The client gets the server RPC and stores the value in "Ts"
  5. Immediately after that, the client gets its own system time again, subtracts "Tc" from it, and stores the result in "Tt"

So now we have three values:

  • Tc - The system time of the client when it sent the RPC request for step 1 to the server
  • Ts - The system time of the server when it received the RPC request from step 1
  • Tt - The total length of time it took for the client to get the server's time

Step 2

Ts was the server's time when it received the RPC; so at the moment the client gets it, the time on the server is actually Ts + (the time it took to send Ts to the client). I'm going to estimate the time it took to send Ts to the client as Tt/2 since Tt is the duration of the entire two-RPC exchange.

Therfore at time Tc, the time on the server was approximately (Ts - Tt/2).

I'll repeat myself because this is important:

Therfore at time Tc, the time on the server was approximately (Ts - Tt/2).

Now that we know this, we can calculate the difference between the server time and client time, and store it in a new value we call "Td"

Td = (Ts - Tt/2) - Tc

Step 3

Now that we know Td, we can calculate the server's approximate time. Since:

Td = (Ts - Tt/2) - Tc

we can add Tc to both sides:

(Ts - Tt/2) = Tc + Td

and interpret the equation to mean:

The server time = The client time + Td

Here are some relevant snippets from my implementation:

cpp code:
/** Gets the current system time in milliseconds */
/* static */ int64 AMagicBattleSoccerPlayerController::GetLocalTime()
{
  milliseconds ms = duration_cast< milliseconds >(
    high_resolution_clock::now().time_since_epoch()
    );
  return (int64)ms.count();
}

void AMagicBattleSoccerPlayerController::BeginPlay()
{
  Super::BeginPlay();

  // Ask the server for its current time
  if (Role < ROLE_Authority)
  {
    timeServerTimeRequestWasPlaced = GetLocalTime();
    ServerGetServerTime();
  }
}

bool AMagicBattleSoccerPlayerController::ServerGetServerTime_Validate()
{
  return true;
}

/** Sent from a client to the server to get the server's system time */
void AMagicBattleSoccerPlayerController::ServerGetServerTime_Implementation()
{
  ClientGetServerTime(GetLocalTime());
}

/** Sent from the server to a client to give them the server's system time */
void AMagicBattleSoccerPlayerController::ClientGetServerTime_Implementation(int64 serverTime)
{
  int64 localTime = GetLocalTime();

  // Calculate the server's system time at the moment we actually sent the request for it.
  int64 roundTripTime = localTime - timeServerTimeRequestWasPlaced;
  serverTime -= roundTripTime / 2;

  // Now calculate the difference between the two values
  timeOffsetFromServer = serverTime - timeServerTimeRequestWasPlaced;

  // Now we can safely say that the following is true
  //
  // serverTime = timeServerTimeRequestWasPlaced + timeOffsetFromServer
  //
  // which is another way of saying
  //
  // NetworkTime = LocalTime + timeOffsetFromServer

  timeOffsetIsValid = true;
}

/** Gets the approximate current network time in milliseconds. */
int64 AMagicBattleSoccerPlayerController::GetNetworkTime()
{
  return GetLocalTime() + timeOffsetFromServer;
}


I'm treating Td as a constant in my implementation. I don't expect the server and client clocks to be running at paces different enough to become significant in the time it takes to finish a game. I also don't want Td to change because the ball movement implementation expects time to always be moving forward instead of going back and forth every so often.

You may also wonder "Why do this from APlayerController and not the ball?" Look at these requirements for clients sending RPC's to the server from

Docs.unrealengine.com :

  • They must be called from Actors.
  • The Actor must be replicated.
  • If the RPC is being called from server to be executed on a client, only the client who actually owns that Actor will execute the function.
  • If the RPC is being called from client to be executed on the server, the client must own the Actor that the RPC is being called on.

The client does not own the soccer ball, thereby failing requirement 4. The client however owns their player controller, and that object meets all the criteria.


Client Movement

During game play the client will get a stream of ball properties from the server. A critical thing to remember is that those properties are always out-of-date because it takes time for them to get from the server to the client. On the client, the ball is perpetually "catching up to where it is on the server." To make the ball do this smoothly, I use interpolation and extrapolation like so:

cpp code:
/** Simulates the free movement of the ball based on proxy states */

void AMagicBattleSoccerBall::ClientSimulateFreeMovingBall()

{

AMagicBattleSoccerPlayerController* MyPC = Cast<AMagicBattleSoccerPlayerController>(UGameplayStatics::GetPlayerController(GetWorld(), 0));

if (nullptr == MyPC || !MyPC->IsNetworkTimeValid() || 0 == proxyStateCount)

{

// We don't know yet know what the time is on the server yet so the timestamps

// of the proxy states mean nothing; that or we simply don't have any proxy

// states yet. Don't do any interpolation.

SetActorLocationAndRotation(ServerPhysicsState.pos, ServerPhysicsState.rot);

}

else

{

uint64 interpolationBackTime = 100;

uint64 extrapolationLimit = 500;


// This is the target playback time of the rigid body

uint64 interpolationTime = MyPC->GetNetworkTime() - interpolationBackTime;


// Use interpolation if the target playback time is present in the buffer

if (proxyStates[0].timestamp > interpolationTime)

{

// Go through buffer and find correct state to play back

for (int i=0;i<proxyStateCount;i++)

{

if (proxyStates[i].timestamp <= interpolationTime || i == proxyStateCount-1)

{

// The state one slot newer (<100ms) than the best playback state

FSmoothPhysicsState rhs = proxyStates[FMath::Max(i - 1, 0)];

// The best playback state (closest to 100 ms old (default time))

FSmoothPhysicsState lhs = proxyStates[i];


// Use the time between the two slots to determine if interpolation is necessary

int64 length = (int64)(rhs.timestamp - lhs.timestamp);

double t = 0.0F;

// As the time difference gets closer to 100 ms t gets closer to 1 in

// which case rhs is only used

if (length > 1)

t = (double)(interpolationTime - lhs.timestamp) / (double)length;


// if t=0 => lhs is used directly

FVector pos = FMath::Lerp(lhs.pos, rhs.pos, t);

FRotator rot = FMath::Lerp(lhs.rot, rhs.rot, t);

SetActorLocationAndRotation(pos, rot);

return;

}

}

}

// Use extrapolation

else

{

FSmoothPhysicsState latest = proxyStates[0];


uint64 extrapolationLength = interpolationTime - latest.timestamp;

// Don't extrapolate for more than [extrapolationLimit] milliseconds

if (extrapolationLength < extrapolationLimit)

{

FVector pos = latest.pos + latest.vel * ((float)extrapolationLength * 0.001f);

FRotator rot = latest.rot;

SetActorLocationAndRotation(pos, rot);

}

else

{

// Don't move. If we're this far away from the server, we must be pretty laggy.

// Wait to catch up with the server.

}

}

}

}


I want to explain the two variables used in ClientSimulatePhysicsMovement():

interpolationBackTime - This variable means "Our instance of the ball is going to be (interpolationBackTime) milliseconds in time behind the server." In my snippet it's hard-coded to 100 because I'd like the average client ping to be at or below that. Why can't we say "well just make it 0 so the ball is always in the present?" Because remember that it takes time for the ball properties to be transmitted to the client; we can't know where it is on the server at the present. If you did set it to 0 then I think the ball would be jumping all over the screen during game play as if to say "whoops I'm supposed to be here, whoops my bad I should have been there, whoops I fell behind again..."

extrapolationLimit - If the server suddenly stops sending data to a client for a second or more, all the client can really do is keep the ball moving in the same direction and hope it's right. You've probably seen objects freeze or "rubberband" in network games; that's because the replication was briefly interrupted on the server and the client wrongly assumed objects were at certain places before new replicated data showed otherwise.

Results

I did get the soccer ball to appear reasonably in sync on LAN clients with this implementation, but have not yet tested over a WAN connection with higher latency. I think there will be some more fine tuning of the code before it's ready for general release. I still feel like I unnecessarily reinvented some wheel here given how advanced the Unreal Engine is though I enjoyed writing and testing the code regardless.

Check out my homepage and social feeds

And my projects!

Groups
Desura

Desura

Official group with 10,095 members, open to all members

Desura is a community driven digital distribution service for gamers, putting the best games, mods and downloadable content from developers at gamers...

Desura Publishing

Desura Publishing

Official group with 186 members, open to all members

Desura Publishing group is a group dedicated to all questions regarding game & mod publishing on Desura. Official features and updates on publishing progress...

Gamieon

Gamieon

Developer with 4 members, open to all members

Gamieon is a small independently owned company devoted to developing innovative and high quality video games.

Gamieon, Inc.

Gamieon, Inc.

Developer Open to all members

Gamieon, Inc. is a privately owned entertainment software development company located in Tampa, Florida. Since October of 2004, Gamieon provides quality...

Unity devs

Unity devs

Hobbies & Interests group with 1,242 members, open to all members

For all Unity developers and developers-to-be, both beginners and professionals!

Post comment Comments
ArtisanCodesmith
ArtisanCodesmith Sep 24 2012, 7:38am says:

I'm new to IndieDB and ModDB and wish to invite a few people (random people picked from the online list lol) to view my game and start generating a few opinions, do you mind taking a quick look? The game is called Arkanius. Thanks, Artisan Codesmith. :)

+2 votes     reply to comment
AlexVSharp
AlexVSharp Jan 30 2012, 6:07pm says:

Wow, well written. You should include this article in the Unity Dev's group. ;)

+3 votes     reply to comment
Post a Comment
click to sign in

You are not logged in, your comment will be anonymous unless you join the community today (totally free - or sign in with your social account on the right) which we encourage all contributors to do.

2000 characters limit; HTML formatting and smileys are not supported - text only

Level
Avatar
Avatar
Offline Since
Dec 21, 2014
Country
United States United States
Gender
Male
Member Watch
Track this member
Accolades
Desura
Statistics
Activity Points
969
Rank
2,132 of 483,204
Watchers
5 members
Time Online
13 hours
Comments
18
Site Visits
2,159
Profile Visitors
70,453 (10 today)
Contact
Homepage
Gamieon.com
Private Message
Send Now
Email
Members Only
ICQ
387263084
Twitter

Latest tweets from @gamieon

Just got wall traps working in MagicBattleSoccer. Need to find a final name for the game before I post much more. T.co

8hours 34mins ago

Got bows and arrows working in MagicBattleSoccer T.co . Thanks @Mixamo for character creation service! #indiedev

12hours 49mins ago

New Magic Battle Soccer Screenshots on Reddit SSS T.co #indiedev

Dec 20 2014, 9:17am

Starting to work on Magic Battle Soccer visual assets #indiedev #gamedev T.co

Dec 14 2014, 9:26pm

New dev journal entry: How I synchronized a simple rigidbody across network players with UE4 - T.co

Dec 9 2014, 5:43pm

Early Magic Battle Soccer prototype on Reddit Feedback Friday T.co #indiedev

Dec 5 2014, 2:03pm

Finally a new Magic Battle Soccer prototype video! New HUD stuff and a weapon that spawns blocking walls T.co

Dec 2 2014, 10:24pm

How I sometimes fix crashes running my Unreal 4 game in DebugGame mode - T.co

Nov 18 2014, 4:38pm

Plotting Paper Cowboys development while trying to get my Unreal soccer prototype network playable. Sorry, no screenshots yet!

Nov 13 2014, 6:03pm

My game studio Gamieon turns 10 years old today! Woot! T.co #indiedev

Oct 26 2014, 10:32am

shaulhadar
shaulhadar friends since Nov 29, 2011
hermesdavidms
hermesdavidms friends since Jan 2, 2011
swiftseraph
swiftseraph friends since Jun 3, 2011
Original_Speeder
Original_Speeder friends since Jun 4, 2011
Namelessness
Namelessness friends since Apr 26, 2013
NullSoldier
NullSoldier friends since Jun 4, 2011