Defend against Reverse Engineering

balaji
By balaji

July 9, 2008

Software reverse engineering is the technique of getting the original source code from the binary. Competitors might use reverse engineering to figure out how you implemented that cool feature. Crackers might use it to see how they can bypass your license policy. Game cheats use reverse engineering, well, to cheat.

Software reverse engineering is the technique of getting the original source code from the binary. Competitors might use reverse engineering to figure out how you implemented that cool feature. Crackers might use it to see how they can bypass your license policy. Game cheats use reverse engineering, well, to cheat. [The table below discusses some of the tools used by reverse engineers.]

Tools of the Reverse Engineering Trade

Dis-assemblers are used to dis-assemble the binary into readable (at least, semi-readable) code. IDA Pro and PE Explorer are two popular dis-assemblers.

De-compilers convert programs from an intermediate language like Java bytecode to their more highly readable original source code. Reflector for .Net and DJ Decompiler for Java are powerful de-compilers. Debuggers are used to step through and trace the flow of programs. SoftIce is the most popular debugger. OllyDbg is another powerful debugger.

Resource monitoring tools are used to track the usage of files and registry entries. The PSTools suite from Sysinternals is a favorite.

How to beat reverse engineering

There are several strategies available to defend against reverse engineering. Code Obfuscation , which we discussed in depth two years ago, is the simplest (and cheapest) method to deter reverse engineers. Code obfuscation changes function names, alters the sequence of code, and adds noise, without changing the functionality of the code itself.

This month, we discuss a more powerful method to beat reverse engineering - using strong encryption with hardware tokens. Aladdin's HASP is a popular commercial solution that uses this technique, and we will be citing many of the techniques it uses in this article.

The approach is simple – encrypt the binary, attach it to a decryption engine and store the encryption key in a hardware token. The decryption engine loads the binary to memory, decrypts it with the key from the token and then runs the binary. Since the hardware token is required to decrypt the binary, this is also an effective mechanism for license control.

Aladdin HASP HL Pro

Aladdin's HASP HL Pro is a popular commercial solution to defeat reverse engineering and protect intellectual property. We shall dive a bit deeper into HASP to see the defenses available.

The Aladdin HASP token comes as a USB dongle that you distribute with your software. The user plugs it into a USB port, and then double-clicks the binary she wants to run. The decryption engine takes over, gets the key from the dongle, decrypts the binary and executes it. The user is unaware of these processes happening in the background, save for a slight delay in loading the program. If the dongle is detached from the USB port, your software detects it and stops running.

HASP offers two levels of protection: the HASP envelope and the HASP API. The envelope contains the decryption engine plus several pre-canned defenses to defeat reverse engineering. The HASP API allows you to embed calls to the hardware token in your code and perform custom checks to defeat reverse engineering attackers.

Envelope Protection

Envelope protection is the simplest to implement and is very powerful too. You can protect your software within an envelope without making any change to your code. Pass your software through a special utility, and it comes out encrypted, with the envelope protecting it. In addition to encryption, here're some of the protection mechanisms the envelope provides:

  1. The envelope implements anti-debugger strategies to prevent an attacker from attaching a debugger to the program
  2. The envelope periodically polls the USB port to see if the right dongle is still present
  3. The envelope implements several code obfuscation strategies, in addition to encryption
  4. The envelope dynamically encrypts and decrypts configuration files
  5. The envelope offers different grades of encryption to the binary: the stronger the encryption, the slower the initial loading

API Level Protection

HASP offers a second layer of protection that can be customized for each application. The HASP dongle is a smart device that can store secret strings, in addition to the HASP key for decrypting the binary. You can store custom secrets in the dongle and then periodically query the dongle to see if it's present. If the dongle is absent, you may chose to exit the program. Remember this is an additional layer of security, beyond the standard envelope. This allows you to check the presence of the dongle from deep within your code. This makes reverse engineering even more difficult. You could even generate noise through red herring calls and confound the attacker further.

Attacks against HASP

HASP and similar techniques using encryption with hardware tokens has been around for more than 10 years. During this period, protection schemes and attackers have played a cat and mouse game. Some of the earliest attacks found weaknesses in the HASP encryption scheme. The next generation of attacks emulated the HASP dongle in software, so an attacker could fool the envelope into believing that a valid dongle was connected. Over time, security vendors have strengthened the defenses. The current generation of HASP has withstood more attacks - there are no publicly described attacks against the latest HASP - and raised the bar against attackers.

References


Tags: Features

About

balaji

SUBSCRIBE TO OUR BLOG

Buyers-Guide-Collateral

WHITEPAPER

Buyer’s Guide to Managed Detection and Response

Download
MDR

Get AI Powered

Managed Detection and Response

MDR-learmore-btn

 

MDR-Guide-Collateral

REPORT

AI-Driven Managed Detection and Response

Download Report
Episode

EPISODE-25

Red-LineAsset-6

Why Your ‘Likes’ on Facebook May Be Revealing Far More than You Thought

Click URL in the Post for the Full Podacst
  • FacebookAsset
  • LinkedinAsset
  • TwitterAsset