Welcome traveller, come rest a while, at the tavern we call the Tales of Yore developer log. You know not what you might find in the dark corners of this dank hollow, will it be adventure? news of a new evil? or possibly tips on where to find the best apples? Ok, I may have been writing too much content recently, but it's that time of the month again. It's a milestone coming up too, Tales of Yore will have been in development for 2 years shortly. If you haven't already:

Play Tales of Yore

(and support on Patreon if you like)

Looks like this is going to be a long one, best get some coffee or coke...

Birthday Boy or Girl...

Yep, its true, on the 16th of the month the game will have been in development for two years. I don't mean it's been released for two years but 16th of April, 2021 was the day the first line of code was written for Tales. It's been active development ever since and open to the public free to play about a week after that. Wow, what a ride. The game started out looking like this:

The door didn't open, nor the house. There was just one zone where people could wander around. I total space of about 1000 tiles to explore. The were no monsters, no NPCs, no animations in the world. Certainly no items, books, quests or mounts. Looking at the stats today...

There are about 490,000 tiles to explore in the world, 885 items, 39 books, 147 types of creature/monster, 194 NPCs and 115 quests. This is before the current update I'm working on goes out. By far the biggest game I've worked on and by far the most challenging and scary. As overwhelmed as I get now and again I thank my lucky stars I started the project and people have deemed it worthy to play.

Its a Drag

I've been trying to address some of the most common feedback/criticism that Tales receives this month. #2 most commented on was the user interface - or rather specifically the dialogs/windows etc that appear in the game. Among a host of updates was the key one - being able to drag windows around on desktop. 

As much as I love not using an engine or framework to build the game this is one of the areas where I lose out, I get nothing in UI for free, it's all hand rolled. I knew this when I started out and since I wanted to support multiple platforms (mobiles/desktops) I decided to simplify the UI so it always acted like mobile. While this was a good choice at the start to avoid getting distracted, this month I decided to address it. 

Adding simple drag wasn't too difficult or time consuming, but working out all the assumptions I'd made based on there only being a single window on a screen at any time was a lot harder. This sort of retroactive update is pretty painful. We had great bugs like you could open the jobs board, walk to the other side of the screen and still view it. Genius!

I am, possibly unduly, proud  of the result though. It's a big complex system and it was possible to rework the foundations (software is not like building a house!) in a pretty short time frame. Players, as normal, were ecstatic for a few days and then forgot that the old version ever existed :)

Over Achiever

Did you know there are 50+ achievements (on and off Steam) in Tales of Yore? I mean, thats a lot, and it's constantly growing. It's really really hard to get them all but its not impossible. Players had asked to be able to see their progress on each achievement. Now given the massive range of things that are used as triggers for achievements this is non-trivial. 

Luckily due to some mistakes I made early in development, I'd already reworked things so that every action a player takes is logged away in the database. The achievements framework had to be reworked pretty heavily so each achievement could have a "progress" function instead of just a "completed" function. 

Numbers Go Up!

Players like numbers. I think thats really clear. They like numbers that go up. Ideally quite slowly but definitely up! This was made obvious by games like Runescape where essentially you spend hours making your numbers go up. I've have lots of feedback that people want to see more stats and numbers where possible. So, another UI tweak - lets show the numbers on the name plate.

As you might be able to guess, doing this while maintaining the tiny size of things to support mobile was not easy. However, I'm pleased with the result. As part of the rework I managed to sneak in a modifiers bar across the bottom (I'm just hungry on the screenshot) - but this lets players see what spells, abilities, etc are in force and know when they're about to run out.

Another part of this was to be able to see your stats more easily when you're in combat. While just a floating health bar was good, players wanted to see everything, everywhere, all at once! (yes I just watched the movie). Another tweak, full bars next to the player:

Event Planner

Another thing that came as a bit of surprise at this stage to me, but one of the moderators decided that running regular events needed to be done. An event in this case consists of getting a lot of players together at one time and them throwing masses of monsters and raids at them until they either die or win out. Much to my surprise they've become very popular!

They've become twice a week and the awesome moderator running them (Holic!) needed a few new commands to make it easier on him. Spawning ad-hoc groups of monsters and that sort of thing. However, the interesting part came when we realised that ad-hoc monsters were affecting drops of rares, so if you attended an event you were almost guaranteed to get a rare next normal fight you did! :)

A few more interesting features came out, like being able to control when you lost all your stuff, so if in an event where you're mostly expecting to die a few times - it's nice if you can get gold and items without risk. It also drove a lot of performance tuning and simulated players - more later.

Lets be Friends

Another "must have" feature, friends! Players have been pushing for this for as long as there has been chat. It seemed pretty important to players to be able to identify and know when their friends were on (the) line. A very quick add and as a side effect got the /block command that lets you block out chat from a list of players. I for one used that function almost immediately! 

The Sims

As part of the event work above I started to look at performance generally. While I knew that CPU wise we could scale up to 1000 players on on the current server I had no real way of knowing what that'd be like in all the other factors that take place on the server. Since Tales is essentially a lock step system, if the frame time for 1000 players gets high it doesn't really matter how much CPU there is - players will still experience lag. 

Hard bit here was being able to test. I considered trying to organise an event where I got as many players online as possible. As it stands with the current player/subscriber count - thats unlikely to be more than about 100 players online at one time. Not really a good scale test. I speculated about this on Twitter and was met with plenty of push back - why don't you just simulate players? Well I had considered it, but I figured it'd be really hard to do with something as complex as Tales. Turns out not so much.

My simulated players aren't quite doing everything. They wander around the world, choosing one of a selection of paths to go. This generates path finding and movement events through the server. The sims are also modelled like players - from experience players don't just click once even if thats enough to get them from one side of the map to the other. They click multiple times, normally a couple of times a second just because that makes them feel like they're moving. So my sims do that. 

Now the fiddly bit, I'm using Node.js - this means essentially my main game loop is single threaded. Argh you cry! How are you going to scale? Well it's not really true - whenever theres an asynchronous process like database or IO - threads get spawned off to deal with those tasks and if you code it right the main thread isn't swamped. Given my tests show that I have plenty of CPU spare even at high loads I don't really need to have my simulated players doing network access or database access - theres no locking in Tales on the database, and as long as my sims are generating the same events as would come from the network the amount of network is a factor of the IO wait on the box (which may or may not be a problem later).

So... now I have limited simulated players. They do everything that normal players do but don't generate database or network traffic. I'm testing the core game loop and nothing else! Nice! So what happened? Well my first test was not good, my estimated 1000 players become 70 - at 70 simulated and real players wandering the world the game lagged to hell. Ooof!

However, since I had simulated players I could reproduce the issue locally (although with my local hardware I needed to increase the player count 10x to get the same effect). This meant I could sit an optimise step by step. After the first round of optimisation I was able to add 500 players in the server with no lag experienced and the CPU hitting a max of 12%. I've continued optimising and I think I'm now ready for some real player testing! :)

Wiki Wiki Wild Wild West

Here's a weird bit of feedback - "I like the challenge, but I don't like that I have to look stuff up on the wiki all the time" - so, you want uh, a challenge that you have to self discover but you also want the wiki to be in game so you can just look things up when you get stuck. Um. Uh. 

Ok then... now every quest has a "Wiki" button. I'm not a fan of putting the solution in game but this convenience button just does a search of the wiki for the quest so anything players have written about it can be found. Theres also now a /wiki command that searches the wiki and displays the results in a new browser window. This is called me compromising, don't expect it often! :)

Keys to the Kingdom

The final and probably biggest change this month - and certainly the #1 asked for feature - keyboard control. Up to this point I'd be worrying about this one, the game was designed with a network model (lockstep-ish) that made me feel like keyboard control would never work. I'd actually tried it a couple of times over the last two years and never got anything close to usable.

I decided that the time had come to either find a way or give up everything. Too many players feeling that without keyboard controls the game just didn't work. It turned out to be incredibly easy once I finally got the idea. 

I've written many networked games and prototypes over the years, so my first assumption that was obviously wrong was that I'd need client side prediction to make keyboard controls in Yore work. WRONG! 

Next up was I'd need to change the network model to support fast response times for keyboard controls. WRONG!

What did I do? Here goes... Tales works on an event queue, the server runs a set of events that are fully deterministic. The server sends these events out to the clients. The clients run along the event queue as they're told to update producing the same results as the server. Over the last few years I've moved the amount of intentional lag in these events up and down and found that players actually find it nicer if the client just tries to keep up as close as it can to the lock step. For most players on the internet they don't experience any perceived lag this way and for those with flakey connections their clients get a fixed amount behind and stay there. In general, no complaints. 

So how do we model keyboard controls as events in the game - my first attempt was to use the existing path based MoveEvents to represent small moves in the world generated by having the clients send their keyboard control state up to the server. This was the approach I'd tried before and unsurprisingly it didn't work. Theres just too much disjoint between keyboard controls and the events that causing moving in that scenario. 

The big jump was when I decided that keyboard control should generate KeyChangeEvents (or boy band events as I call them). The key change events are only delayed as much as anything in the world (i.e. not a lot) and the simulation reacts to these events by moving the player on the tick of the game. 

Why is this better? First, it is, players like it way more and it feels natural when you're playing. Second, it's because the delay is in the change to your keyboard events not to the effect of them. The player moving is enacted by the simulation so that continues happening whether the key change event is delayed or not. This just feels "normal"

All in all I was over the moon to finally have a system that worked. It was at that point of course I found the 10^6 issues that came with being able to move on the keyboard rather than via long distance path finding. A week or so later at least, we're back to stable and happy!

The End....

It's been a long time, getting from here to there, the last two years have been awesome both from meeting new people, building a community and learning tonnes about game development and games business. I look forward to the next years of work - some days are harder than others and I really appreciate all the support from everyone. 

Journey Onwards.... (again)