Welcome back to the Tales of Yore Dev log! The last couple of months have been pretty slow paced and I even missed a devlog last month while trying to get back on track. However, this month, I'm back in the groove and things are progressing at a hell of a rate. August has been a very productive month and I'm now on track for the Northlands release in September! So, lets go!

If you haven't already, then:

Play Tales of Yore

(support on patreon if you like)

Marketing Makes the World Go Round.... not

In game development circles people talk about marketing as the holy grail. How do we get our games in front of people who might want to play them? We talk about social media and review sites and Reddit. We obsess over how to make the right content to gain the right attention and the right momentum. This month brought the conclusion of months of planning for a marketing push and I'm here to tell you most of those mediums aren't science, or predictable, it's dumb luck. Here's what I've found (which may be entirely obvious) and of course YMMV:

All of that said, I did manage to get a serious increase in players this month. It wasn't to do with any of my well laid marketing plans. It wasn't because I paid my way through. What actually worked?

Channels. I say it again. Channels!

I went from early access to released version on Steam. This gave me a "visibility round" on the Steam pages, I was top of the list of new free to play games for a week or so. The impact was huge. Constant stream of new players. What's more the players coming in knew what they were coming into and were the types of people who would actually play the game, and eventually pay for it - through Steam. The point being that you already know where the players that might actually pay for your game are - they're on the places where they already play and pay for games like yours.

I'm going to go do the same on Google Play and Apple App Store, once I work out how!

Local things for Local People (Performance Part 1)

The Tales of Yore game server is hosted in Europe. It's hosted there because thats where I am, not a good reason. However I now have players across the world and some of them struggle to get reasonable connections. I'd been meaning to address this for ages, so this month I got down to it. Things to consider:

• I don't want to run multiple separate servers, I don't have a user base that supports that sort of separation - their one community together.
• All network communication for Tales goes over web sockets.
• I have players from Asia and North America who are currently relying on public internet routing to reach the server
• I can't afford to create massive servers for this, the game server costs about $50 a month, I can afford maybe $10 a month extra in overhead.

The key thing here is to optimize the path from the client to the game server. From my day job I know a bit about these things, including that running over a hosting providers backbone is generally a bit faster but also more importantly more consistent in terms of timing. So a public internet route from Asia to Europe might take 150-250ms and vary wildly between the extremes. A route on a hosting provider for the same link is likely to be towards the 150ms and remain consistent. So... lets get clients connected to the hosting provider network as soon as possible and then routing across the backbone.

So whats that then? The boxes on the left right are servers geographically close to my players. One in Asia and one in North America. The public internet route from player to these boxes is likely to be quick since it hasn't got to go far. Once the player's connection reaches this point its then routed across the hosting providers backbone.

Now, because I'm using simple web sockets I can do this using an out of the box solution called NGINX. NGINX is a very configurable proxy server. These days it's also used for hosting and serving files too but it's original job was as a proxy, or in this case a reverse proxy since its on the inside of the network not the client side. When a connection comes in to the access points it's simple accepted and forwarded on to the main server. Since this is such a simple process the access boxes can be very very cheap VPS boxes (more on why VPS is painful later), a $5 a month each. We win!

While proxy is optimal to use is automatically detected by the game client and players are now playing using these access points in production to great positive effect!

It's Bugging Me

As always theres been a raft of bugs to fix, here's some examples:

• Crypt Level Markable - for a short while it was possible to use recall and mark inside dungeons. While this wasn't a big deal the discussion around how people were using it has resulted in a change that lets mounts enter dungeon (unmounted). This in turn has resulted in the need for extra restrictions, in this case pet food. Funny how these things flow.
• Furniture Place Outside - Somehow we got to a state where people could place furniture outside their house. This resulted in some hilarious setups that I felt terrible about fixing.
• Boosts Cause Lag - Server boosting is very popular, players pay in game gold to increase the exp or gold gain rate for everyone on the server. It's particularly cool during events. However, a player remarked that while a boost was on they were lagging. They were convinced that it was related. As normal I was 100% sure it couldn't be related until I realised it was! 

Sharing is Caring, and slow... (Performance Part 2)

I've been working a lot on performance this month with a view to seeing player counts increase. However, players have been reporting small lag spikes on a regular interval for a long time now. Tales of Yore, for better or worse, works on a fixed frame loop. If this frame loop takes more than about 60ms for more than about 30 frames then the players are going to feel it. As part of the performance work previously I'd added a custom game frame profiler so I could see what was going on and I worked and worked to try and improve the performance:

• Found a bug that mean that empty zones were being updated
• Tuned the hell out of path finding
• Reworked the update loop so it didn't have to scan zones to look for which ones need updating
• A hundred other tiny improvements

While this was getting somewhere I was still getting very confused over some numbers. I'd have simple pieces of code like this:

if (now.getTime() - this.lastPing > PING_INTERVAL) {
    // this is an asynchronous call, it'll return instantly and 
    // and send when IO is ready
    this.protocol.ping({
        time: Date.now()
    });
    this.lastPing = Date.now();
    this.receivedPingResponse = false;
}

Most of the time this would report as zero milliseconds (or 0.01 milliseconds) for every client this happened in. However, I could see that sometimes, even when only one player was connected and hence this was only run once per frame, this could take 300+ milliseconds. How the hell is that possible? This code should always be quick. It's not really doing anything?

To work it out I had to come up a level. It wasn't the code but the hosting. For financial reasons I've been running Tales on some reasonably cheap VPS (Virtual Private Server) servers. As we can see in the diagram above I actually have two, one for the game and one for the database and website. I thought this would be most cost effective however VPS servers have a downside, they're virtualised (clue is in the name) - this means you're actually sharing physical hardware with other people. You each have an allotted CPU usage. Unfortunately this means that your performance can be a bit "spikey", if another VPS on the same physical hardware as you decides to do something a bit heavy (but still within their allowance) then you might find you get paused. 

Ok, so this was my theory but I needed to prove it. As mentioned above my cost is about $50 a month for the main server and dedicated hardware hosting is more expensive. I can't afford to run two boxes on dedicated hardware - a single box is going to cost me about $50 a month. I make the decision at 9am that I'm going to go to dedicated hardware, combine the two boxes I currently have on to one machine and migrate everything across. 5 hours later I have the hardware provisioned, I have a plan written up to follow when I do the migration and I have warned players that a migration will mean the game will be down for an hour or so. I go for it! Less than an hour later the DB, game server and website have all been migrated!

Well, what was the result?

The spikes are gone. The performance improvement is 5x. The new hardware costs me around $5 a month extra and I can now profile performance correctly - no oddities reported at all. Game is now running with zero lag and I can predict the scaling. Generally, awesome!

Next step is to work up a multi-box strategy so i can scale across multiple processes (to use more than one core with NodeJS) and eventually across multiple servers. I have a plan now and I desperate want to go play with code but I have to finish Northlands first!

Boss Behaviour!

Lets get back to in game stuff, this month I added boss behaviours into game. This allows the new bosses in Northlands to have unique capabilities. One boss can create walls of fire. Another makes rocks fall from the dungeon ceiling. Yet another will chase you then give up and use magic instead.

Bosses up to now have been limited to single types of attack with the exception of one that could change back and forth. The new bosses are smarter and more dangerous. Good luck players!

Content is King

This month I've made huge strides in the content for the next big update, Northlands. It's been 6 months of work so far and it looks like it'll be about another month before its done. I've added hundreds of items, weapons, equipment, harvest points. There are another quarter of a million tiles of game world to explore. Loads of new secrets, monsters and bosses. 

Worth noting that player's in general really don't care about all the technical bits and pieces being done, or the struggles of marketing etc. Remember this, what they want is MOAR game! 

Quality of Life

Listening to players and delivering what they want/need is one of my favourite bits of software development, including my games work. It's really great to understand how players are making use of the game features and how they've worked out how to beat it. This month brought a lot of quality of life (or QoL as players always say) improvements to the game.

Players often come forward with a problem and a solution. Sadly the solution is often either too complicated or too game breaking to implement. The really interesting bit is talking to them to understand what the problem actually is, and then solving it the right way. 

Lost in Translation

I've always been keen on making the game available to as many people as possible, partly because I like the idea but also it creates a bigger market potential for selling. As such Tales is translated into a bunch of languages (French, Chinese, Spanish, Italian, German, Portuguese, Russian, Korean, Czech, Japanese). Since it's only me doing this project and I don't speak anything other than English, the translations are generated through DeepL and Google. 

Of course some players aren't always happy with machine generated translations and I've often received offers of help to translate the game. This month I made it possible for people to help out with a github project for the translation files: https://github.com/kevglass/talesofyore-translations

Anyone with the inclination and time can go through the files and add either better translations or a completely new language. I validate them through the auto-translate on the way back in and we're good to go. Both Czech and Chinese have been completely retranslated by players through this mechanism!

Moderators are the best!

I'll go ahead and say it again, my moderators are the best. They work really hard to keep the game family friendly and find the bots that get past the inbuilt bot checks. They also run epic events which keep the players entertained. What's more they're all active players with some of the longest time players (I'm looking at you D1sk! 134 days of game time and counting!)

This month I've been trying to make their life easier with a few new commands and options. First are the links commands that allow people to put links to specific sites into the chat. The game tries to prevent people posting links generally to prevent spam but there are a select of links (Discord, Wiki, FAQ) that would be useful to post. There are now commands to create simple embedded links in chat:

The next was the ability for moderators to boost the server without spending their own gold. Up to now moderators had been spending millions of in game gold to boost the server during events. The admin console that they have access to now allows them to boost it directly without having to spend their hard earned gold!

Giving the Gifts

The final update to mention this month was something asked for repeatedly by players, the ability to gift each other cosmetics and subscriptions. The community in Tales is extremely friendly and supportive. As such they like to help each other out where they can but to give each other these things they had to come through me. Not any longer, you can choose to buy either a subscription or a cosmetic as a gift for another player!

Well, you made it this far, I'm impressed. Suffice it to say its been an extremely busy and productive month. I feel like I'm finally back into the flow of development and making great strides.

Thank you for reading, if you have any comments or questions drop them in the comments. Until next time, Journey Onwards!