Crysis 2 Design – Console to PC

I bought Crysis 2 during the recent Steam Summer Sale. I’m actually quite proud of myself for only buying one game since I have never even installed more than third of the games I own, but I knew that I’ll finish Crysis 2 pretty quickly. I was a huge fan of Crysis 1 when it came out, especially since I bought a state of the art gaming computer in 2008 and Crysis was the benchmark. The first play-through was an amazing experience graphically and gameplay wise. In my opinion Crysis 1 was the most innovative and awesome FPS since Half-Life 2.

It took me a week to play the 18 hours it took to finish Crysis 2 and I wanted to write about the design of the game and the gameplay differences between the two games. I’ll start with a sweeping declaration that Crysis 2 was designed to be a console game first while Crysis 1 was designed to be a PC game. This will be a recurring theme today and I don’t care if it is an overused, old and corny observation.

A computer showing an old TV rainbow test screen.

Is that how people in Crytek think computer monitors behave?

The first indication of it being a console title is the “press any button to start” screen, an annoying annoying reminder that I’m playing on the wrong platform. The first gameplay difference was the checkpoint system and the lack of quick saving. Quick saves are considered a PC feature because of disk space issues on consoles. Quick saves have to describe the whole state of the map and all the information about the player and his enemies which adds up to a lot of data on games with a lot of moveable items and destructible environments. Checkpoint saves only need to record some data about the player because they are placed at locations where the state of the game is controlled (for example you can’t return to the last area and see the destruction you caused and you still can’t do anything to affect the area ahead). On consoles you have less disk space and more memory constraints so the more data efficient solution, checkpoints, is used.

Quick saves are not the only place where the console design makes this a worse PC game. Hard mode is another example. In Crysis 2 hard mode is not hard, just tedious. Crysis 1’s hardest mode was awesome – the enemies talked in Korean so that you could not understand their intentions and strategies, your crosshairs were removed forcing you to use the various scopes and iron sights and there was no longer a huge red HUD warning when a grenade is thrown at you (among other things like making enemies stronger etc). Hard mode in Crysis 2 doesn’t have any of the creative features above. They just made more enemies and gave them more health. Tedious. I presume these changes are necessary because removing crosshairs is a huge cripple for people playing with stick controls where it is harder to aim and not showing grenade indicators is bad for people playing on bigger displays with less detail where grenades are harder to spot.
To continue the list of down grades in the game:

  • You can’t ruin the environment as much. Another performance issue – physics engines with dynamic environment destruction are just slower and have a bigger hit on the hardware. Coupled with the amazing graphics it is probably beyond current gen console’s abilities.
  • AI sucks. I guess it wasn’t too good in the first game, but the baddies in Crysis 2 just love to chat on their open radio com telling me exactly what they are doing and where they are going. C.E.L.L really needs to invest in some scrambling tech.
  • Soldier standing looking at a wall with his back open to attack.

    Apparently this soldier is so afraid he decided to literally hide in a corner.

  • I feel there is less variety in killing enemies. They did add stealth take-downs which are very fun, but they took away a lot of environmental ways to kill enemies (see point one). One of the most memorable moments in Crysis 1 is collapsing an outhouse on a soldier, killing him in a most uncomfortable manner. Crysis 2 had none of that. Even picking up enemies and throwing them is less fun.
  • Less modes – strength and speed modes on the suit were combined into the jump and sprint abilities and their effects lessened. I guess this is done to make the game easier for people who have less than 10 buttons to work with. I think that as a design decision it makes it easier to play but again removes variety which is bad.
  • Less visual and locational variety – it might be because of the location; New York city is not as visually varied as a tropical island with a spectacular alien cave and a frozen side. The visuals are actually very good, colorful and are a notch above the usual brownish of modern FPS but still not as good as Crysis 1.
  • I feel like the story got in the way of playing while not being good or memorable. In Crysis 1 there wasn’t much story which also meant less immersion breaking cut scenes and unskippabale sequences which is pretty good. I can write a whole post only about that, but Shamus Young already did and I agree with pretty much everything (also don’t miss the second part about player volition).

Despite all of that, I really loved playing Crysis 2. After modding the game to include quick saves, I must say the basic Crysis formula is still there and is still as fun. I really love some of the new features like the silencer attachment not taking you out of stealth. Also, detaching mounted machine guns really lets you feel like the superhero you are, especially combining it with armor mode and just walking in the open mowing down enemies like an uber-charged heavy from TF2. It’s pointless to comment on the graphics because the Crytek engine is synonymous with amazing views, even on a 4 year old PC like the one I was playing on. Another thing I liked is that the aliens in this game were pretty interesting and had some scary abilities like jumping from place to place, charging and EMPing the energy and cloak away. These could have been a good experience if only the enemies themselves weren’t dumb as hell and pretty much on rails for most of the time.

Using the environment to guide the player forward.

Using the environment to guide the player forward.

To summarize I enjoyed the game about half as much as Crysis 1 but it still was a fun experience. It is also interesting to see how the series is moving away from its PC start and into Console land (with PC as a secondary platform) and the gameplay consequences of this business decision.

Practical Programming: Technical Debt

This is part of the “Practical Programming” series in which I try to say something profound about the craft of programming.

Technical debt is a very important factor in any software project yet you might never even encounter it until you start working in the field and having long-term projects. Even then, technical debt is something that creeps up. It’s not a bug, nor a feature you forgot or a meeting you dozed off in. It is more of a feeling, a state, a cup that slowly fills up until it overflows.

My definition of technical debt is the difference between the code right now, and the code if it was written perfectly by a thousand coders with unlimited time. This debt includes missing documentation, no tests of any kind, no release process, no design, no consistency in the code and other such issues that are usually regarded as secondary to actual working code. The metaphor is aptly named because it also has interest – the more the technical debt grows, the harder it is to scale development, be it adding new features, adding new developers to the team or being efficient in solving bugs. In the end you go broke and declare code bankruptcy – a rewrite.
(Some great resources that explain it in more depth: Martin Fowler’s article, c2 wiki page)

Let’s unwind the stack of metaphors and get back to why is technical debt a problem – to create a functional program all you need is code that compiles, but to create a sustainable development environment you need code, design, documentation, tests, deployment procedures, etc. This difference is the technical debt of your project. If a lot of those basic things are missing from your project it’ll get tougher and tougher to make even the smallest change without ruining something. As with any debt the interest will keep on climbing until it will be obvious that a rewrite is easier than fighting the code. This step always comes. You can ignore it and expect developers to work around these problems but eventually this will stop being cost efficient – bugs will pop up everywhere taking time from developing new features and new features will be buggy and produce unexplained behavior (the old “compile it and let’s see what happens”).

There are several ways to lessen the burden of technical debt. The easiest is just slowing down the development process and allowing time for basic software development practices. This is definitely the preferred way to do your projects (hehe). The other side of that is rewriting the whole code base, adding 1 to the version number and starting a  marketing campaign – “new packaging”.

I’ve recently had to think about it and I’m advocating a hybrid solution – allocating time in each development sprint to pay the technical debt of a specific class or package in your code base. I have summarized my approach to five easy steps:

  1. Documentation – create a page for the package using whatever documentation repository you use. Document the following about the code:
    1. How should it work? This means a general overview of the correct working procedure for the code and comments, if there are any, about how the current implementation differs from this ideal.
    2. Class level documentation – a few words about each classes’ responsibilities and the interactions with other classes. If there are more than 3 classes a diagram might be needed.
    3. Other things that will need to be documented: XML and other data exchange formats, client server interactions, security measures, relevant databases, etc.
  1. Code Documentation – Document the code itself:
    1. Documentation for classes and all public functions. Use your platform’s most popular documentation framework.
    1. Document algorithms and private functions as needed.
    2. Add a header to the file. A header should contain who is responsible for the code (usually the author), a date and a short description of the file’s content. Some places will also want the license added to the code.
  1. Uni-test the code – Write at least one unit test for every public function in the class. Try to hit the major code paths. This will make refactoring easier because you’ll have the safety net of knowing the tests pass, and having one unit-test makes the mental barrier of adding more way smaller.
  2. Make easy and safe refactorings::
    1. Magic values into global constants. Strings to external files for easy L10N
    2. Don’t Repeat Yourself (DRY) problems – if some code is copy pasted make it into a helper function / its own module / whatever. Sometimes this needs to be said… I know.
    1. Run Lint on your code and see what is easy to fix.
  1. Code Review – Review the code and document the major changes you’ll need to do to make it more robust. Create tasks, put them in the backlog and allocate time for them to be fixed along-side bug fixes and features. This might seem like cheating because you’re still deferring the problem, but knowing what needs to be done is half the battle. If you have it among your other tasks it is easy to schedule it and consider it part of development instead of some concept that doesn’t contribute to the push forward.

Is there a way to write software without getting in debt? Probably, but it might not be practical. Let’s not have perfect get in the way of good enough and ask is there a way to write software without getting into much debt? Of course. The best way is identifying those moments where you are deciding between a “quick and dirty” solution and a slower but better solution, and understanding that the “quick” in “quick and dirty” is only short-term and it might be slower in the long run because of the effects of technical debt.

Reflections: UnUrlShield – Fighting CAPTCHAs

Update: A challenger appears. My security researcher friend Fox has challenged me to a duel. See her blog post for the details.

This is part of the Reflection series in which I go through my old projects and tell their story.

CAPTCHAs have become an integral part of the web in the last few years. Almost everyone on the web has encountered those twisted pictures, probably when signing up to an email service. They come in various shapes, sizes, colors and cats. When they first became popular, there was an explosion of different types of schemes that services used (who can forget Rapidshare’s cat captcha?).
Now as with every security measure there is a compromise between usability and protection. Some of the easier CAPTCHAs were broken using only OCR software, while some of the latest reCAPTCHA images are hard even for a human to solve (interesting but out dated chart).

Various captchas from wikimedia.

One such service was UrlShield. You would give UrlShield a URL you want to protect from bots and it created a page with a CAPTCHA that when solved correctly redirected you to your original URL. Simple enough. I can certainly see a use for such a service, for example if you want to give out promotional coupons and don’t want bots to snatch all of them. The service became popular in some file sharing sites for the same exact reason.
The particular image this site was generating had a checkerboard background with 4 characters all in different colors, sometimes overlapping. It was pretty easy for a human to parse it.
Example of UrlShield generated images and their OCR.
It even works pretty good against OCR. I used Microsoft OneNote OCR feature which uses the commercial OmniPage software to create the second column.

So far so good? Well, no. This scheme is flawed because it is easy to transform the image – remove the background and segment it (split it to region that each contains a single character), allowing OCR tools to easily get the letter. To remove the background you just clear all the black pixels out of the image. To segment it all you need to do is choose one color and mask all the others, which means you’ll end up with a single letter, as each letter is in a different color. This is what you end up with:
Parsed captcha images
OneNote has no problem parsing each of these to a letter.

The process described above is exactly what UnUrlShield does. It’s a simple Python script that use the Python Imaging Library to read the image. Then it counts all the colors that appear more than a certain threshold (MIN_PIXEL_PER_LETTER_COUNT) and saves each color’s pixel location. Lastly it goes through the colors, creating an image with only that color’s pixel locations.

Is there a lesson here about CAPTCHAs? I think so. UrlShield is now some kind of ad/malware site. Even complicated CAPTCHAs can be broken, or even better – be defeated by side-channel attacks like having an army of low-cost workers break them on-demand (The comments of this article are a treasure trove of irony) and sometimes people are even fooled into breaking CAPTCHAs. This is why it amazes me they are still around, annoying normal regular people while also being broken by even slightly motivated attackers.
Are there no solutions to spam? Of course there are! In fact gmail does a great job at stopping 100% of my spam using things like blacklisting known spammers, Bayesian filtering, “crowd-sourcing” protection (the “mark as spam” button) and other tools that don’t rely on CAPTCHAs.

Do you have good examples of silly, easily broken or bizzare CAPTCHAs? Did you find an easy way around some services blocked by CAPTCHAs? Leave a comment below and tell me about it!

Practical Programming Intro

Today’s blog post will introduce another series in this blog – “Practical Programming”. This series will be about developing software in the real word and about subjects that you won’t usually see in computer science curriculums.

What do I know?

Well to be honest – not much. I started a degree in high school but basically flunked out by the third semester, and never continued it. I’ve had a few professional courses in programming, but most of my knowledge and experience comes from on the job training, side projects (see my “Reflections” series), the internet and other unofficial channels over about 7 years of employment as a developer. I’ll also concede that 7 years is basically nothing, and I’m at least 3 years from even starting to master the field. So why the hell am I even qualified to write about it? Very simple – it’s my party, and I’ll write about what I want to.
Seriously though – it’s up to you to decide whether I am making sense and whether to learn from or discard what I write, or even better – start a discussion and give me your opinions.

Why is that important?

Programming the science is pretty different from programming the craft. To be successful at the craft you need way more than a mastery of a particular programming language or platform. Some general skills like teamwork are a must but also understanding project management concepts and methodologies (waterfall and agile for example), quality assurance concepts and a few other essential skills. The sad fact is that being a “rockstar” (eww) programmer is dependent more on one’s ability to ship software than on one’s skills with C.
This series will talk about the skills you need to ship software.

Some of the topics I already lined up for this series are:

  • Technical Debt
  • Continuous Integration
  • Python’s “There should only be one obvious way to do it”
  • Time Estimations

Hopefully you’ll find this series interesting and informative, and you’ll become a better developer because of it.

Reflections – an Intro and an Example

In this first post in the “Reflections” series I’m going to explain what is the purpose of the series, talk about the reasoning behind revisiting old projects and finally applying it all to a small but fun piece of code.

Reflecting on What?

This series will be about reviewing my github projects. This serves the dual purpose of having at least 10 topics for blog posts and writing documentation for old projects in the hopes that someone else finds them useful or interesting. About a year ago I’ve decided that I must do something about the tens of “side projects” that I have from more than 5 years of coding. I quickly triaged the list of about 25 projects into ‘forget’ or ‘open-source’ and just threw everything presentable with a really minimal readme to github.
Part of the list of projects

So why write about it?

  • Code isn’t as find-able as text. To make this code useful, it needs to be at least somewhat documented. When I uploaded them I half assed a read me file but I doubt that is enough for a search engine to index it correctly.
  • I think most of the projects has something unique to add to the web and are interesting both at the code level and in the research that brought them to life.
  • Revisiting old projects will give me a nice nostalgical feeling.
  • Did I mention 10 easy blog posts?

What am I going to cover?

  • The why – The research that went into the code. Since most of the projects are a Proof of Concept (POC) I think it will be interesting to remember what started that particular train of thought.
  • The how – The parts of the code that do interesting stuff and how do they do it.
  • The what now – Maybe a paragraph or two about how the project can be productized or improved and where can it evolve from here.

Let’s try this out

To finish this intro I want to talk about a small but fun hack I did when I got my first Android phone. It was the happy era of 2.X Android and I got a HTC Desire from England. It was right about that time that they announced that next version will be out in a month (Haha, I still believed OTA promises) and will have Hebrew support. That meant that I had a month where my phone didn’t have the Hebrew language font and most importantly I couldn’t read any SMS. Since the phone was new I didn’t want to experiment with rooting just yet, I needed to find a clever solution. One of the first apps I installed on the phone was the Android Scripting Engine (also known as Scripting Layer 4 Android, unfortunately development looks pretty dead now, although it works on my Galaxy Nexus) since I hoped there would be a way to write real apps with it. Unfortunately it was quite minimal, but one thing it did have is a way to get your SMS. That meant I could write a simple script to transliterate Hebrew Unicode into English (ASCII).
Picture of the SMS transliteration app

The code itself is very straight forward. It is only a few tens of lines, and there is pretty much no reason to ever use it again, but I think it represents why I like the Android – it has more of a “hacker” spirit than the iPhone. The only interesting thing I see in the code is that the sign for NIS and the u2029 whitespace in the translation dictionary. I guess I started manually adding characters in an attempt to map Unicode to ASCII. Finally I gave up and added the error parameter to encode that puts a “?” character for every Unicode code point it can’t print.

Why Start a Site and Blog?

To have my own unique place on the web to speak my mind and to reach an audience.

My own

There are plenty of places to speak your mind on the web – social networks, blogging platforms, wikis and the list goes on. The one thing a lot of people don’t realize is that most of those places control your content, your posts, your tweets.  Sure, some might do it less blatantly then others, some might say that they don’t censor and some have an export option, but the biggest problem is that more often than not they also control your audience. I’m not against such services (twitter is even starting to grow on me), but I don’t like the lack of control. I want my online persona to be under my own domain, designed (even poorly) by me and controlled by me.

Unique place

Continuing on the same note – is unique. Not only is that my nickname for years now, but it is also a great domain name and more importantly, brand. So yeah, it will take a lot of time, content and luck for this brand to get big, but it will be more rewarding to me than having the same reputation on twitter or Google+.

Speak my mind

Doesn’t everyone want to be heard? Isn’t attention one of the most basic needs of humans? Less philosophically – I’m going to use this space to broadcast my opinions, my knowledge and my speculations. But like everything in a pragmatic mind – those might change, become more accurate and be proven wrong by reality and by the audience. Topics for this blog include computers, programming, developing, computer games, bicycles and everything else I might get interested in.

Reach an audience

The last point, but probably the most important – while I can control what I write here, no one can really control who views it, who it reaches and who has something to say back. This blog is pretty much like every other product – you put it out there and hope that it helps someone and that someone will get passionate about it enough to start a conversation.

A sign of things to come

I mentioned the general topics I will be covering so to get me a bit more committed here is a list of upcoming subjects for posts:

A series about my github projects – since they are already out there I want to have a post or two about each detailing their history, code reviewing some stuff and capturing the knowledge in them in a more googleable way.

“Practical programming” – a series about basic modern software developing concepts that are usually not taught at school. Example topics include:

    • Technical debt
    • Agile practices
    • “There should be one– and preferably only one –obvious way to do it”
    • separation between view and logic and more and MVC

I’m gonna try and have at least two posts a week.
If that sounds interesting you should subscribe using RSS and be sure to comment and talk back so that I can improve – this side of the keyboard is new to me, so let the journey begin 🙂