Light in the Dark: How OSINT can Counter Crime on the Dark Web

It’s broadly understood to be the murky underbelly of the internet, yet despite the Dark Web’s infamy, it is perhaps not fully understood by the public at large. While not necessarily illegal in itself, this clandestine domain is nonetheless an undeniable hotbed of illicit activity – a trait which reflects the anonymity it affords its users.

Afterall, criminals tend not to parade their doings in plain view, but rather depend on secrecy to ply their trade. As a result, black markets are booming on the Dark Web, where dubious merchants think their nefarious activities can go unnoticed. However, modern open-source intelligence tools are proving this to be a misconception.

• Defining the Dark Web
• The Dark Web’s Legal Status and Usage
• The Evolution of the Dark Web
• Darknet Marketplaces and What They Contain
• TOR: Gateway to the Dark Web
• PGP Key Anonymization
• OSINT: A Torch for the Dark Web

Defining the Dark Web

So first things first – what exactly is the Dark Web?

It can help to think of the internet as being composed of three sections: the Surface, Deep, and Dark Web. As a rule of thumb, the Surface Web can be viewed as everything you can access and see via regular browsers, and the Deep and Dark Web as the online territory invisible to Google and others. But here’s more explanation.

The Surface Web

This is the part of the internet which is totally accessible for the public and can be navigated using standard search engines. In other words, it comprises all the web pages and sites indexed by Google, Yahoo, Bing, etc. – for instance, the web page you are looking at right now.

With Google alone having indexed up to 50 billion web pages as of 2022, you may be forgiven for assuming that the Surface Web accounts for the vast majority of the global datasphere. But – fun fact – it only amounts to approximately 4%. The remaining 96% constitutes…

The Deep Web

That’s right. It’s over nine tenths of the world wide web. In contrast to the Surface Web, the Deep Web – also known as the Hidden or Invisible Web – is the portion of the internet which is not discoverable through standard search engines. This includes all unindexed web pages, password protected sites, private databases, intranets, and the Dark Web.

For example, all fee-for-service (FFS) sites store content on the Deep Web, so only paying subscribers can access it. Netflix is a case in point. Also, sensitive databases – for instance those enabling people to manage their finances online – are of course submerged in the Deep Web along with all closed or private networks.

The Dark Web

The Dark Web is actually just a subset of the Deep Web, and comprises many unindexed sites which can be explored in a similar fashion to the Surface Web – except specialized browsers are required. One such example is TOR, through which users can navigate from site to site without ever surfacing onto the Visible Web.

Another defining characteristic is the complex encryption which pervades the domain, providing a cloak of anonymity to its users. Highly randomized network structures generate “virtual traffic tunnels,” making user activity extremely difficult to track. The Dark Web is heavily concealed and can be thought of as the deepest part of the Deep Web.

The Dark Web’s Legal Status and Usage

The Dark Web has almost become a byword for illicit practices. But is accessing this internet domain actually illegal? The short answer is no, but there’s more to it than that.

The Dark Web is just a piece of technology, but of course the way it is used can be either legal or not. In the same way that in the US it’s perfectly legal to own a gun, but at the same time, any number of crimes can be committed with one. Likewise, the Dark Web can be benign or harmful depending on what users choose to do with it.

Due to its anonymous nature the Dark Web is awash with marketplaces for drugs, stolen data, and all manner of weird and blood-curdling trade for which it has gained its notoriety. Yet there are also legitimate roles that the Dark Web can play, especially for those whose lack of privacy could be hazardous – political dissidents for example.

This double-edged nature of darknets places them in an ethical gray area, and the percentage of the Dark Web being used for illegal ends is not a simple figure to determine. However, one study has found that 57% of live dark sites contained illicit content, suggesting that darknet criminality is a vast and significant issue.

The Evolution of the Dark Web

Emerging in the early 2000s, the Dark Web was originally developed alongside Freenet – a platform created to protect users from government surveillance and cyber threats. As darknets evolved, they took on increasingly sophisticated algorithms for masking IP addresses, making it extremely difficult to determine which sites a given device had accessed.

Then with the advent of Bitcoin in 2009, darknet marketplaces exploded. The combination of a cryptographically concealed platform and an anonymous digital payment system allowed markets on the Dark Web to flourish, attracting a huge influx of both vendors and customers. By 2020, income from darknet markets had exceeded $2 billion USD.

Darknet Marketplaces and What They Contain

Considering that these marketplaces are the go-to platforms for buying and selling illicit wares, the type of items being traded are limited only by human imagination. Yet, despite their shady and exotic nature, darknet marketplaces operate – by and large – much like their mainstream counterparts.

Similar to legitimate e-commerce sites, vendors upload details on the items including photos and descriptions. Customers dip into their crypto wallets to make the payment, which are often held by the marketplace admin until the goods have been received. Like on Amazon, buyers can leave reviews and sellers have ratings – something traders generally aim to keep high to show that they are ‘reputable’ merchants.

So what kind of things are up for sale? Anything you’d care to imagine. Contraband such as drugs and firearms are par for the course, but leaked data is also big business. A 2021 SpyCloud study reported 25.9 million Fortune 1000 business accounts were doing the rounds. What else? Uranium, lifetime Netflix accounts, plastic explosives, fake passports, or – if you fancy – a hijacked private jet!

TOR: Gateway to the Dark Web

Standing for The Onion Router, TOR is the most popular software used for exploring the Dark Web. It’s a free, open-source web browser that preserves user anonymity by routing  internet traffic across various IP addresses, or ‘hosts.’ There are nodes in the network that transport your activity, with a portion of the encryption being peeled away during each operation. Users essentially carry their activity with them to avoid leaving trails.

TOR’s immense rise in popularity is reflected in the browser’s bandwidth which has seen a dramatic increase over the last decade. From practically 0 Gbit/s in 2010, TOR’s traffic had reached a huge 400 Gbit/s by 2021. As of 2019, the browser had 2.5 million daily users and 7000 active routers worldwide.

PGP Key Anonymization

While TOR provides users with a significant level of cloaking, it is not without its issues, prompting many a darknet player to employ additional anonymization methods to mask their activities. Central among these is PGP key encryption, which enables interlocutors to scramble messages so they are only decipherable for the communicating parties.

Here’s how it works. Say User A wants to send User B a totally private email. For this purpose, User B needs to create two keys: one public and one private. The public key provides encryption, while the private key is the decryption counterpart. User B then sends the public key to User A for encrypting the message, which is sent to User B and decrypted with the private key.

In short, anyone with access to the public key can encrypt an email, while only the owner of the private key can decrypt it.

OSINT: A Torch for the Dark Web

The Dark Web is an absolutely crucial source for generating high quality intelligence, and can be leveraged for various key purposes. With so much illicit activity playing out in darknet space this stands to reason. Yet, despite this, it is also a source which is frequently overlooked by intelligence and security departments.

While anonymity looms large in these obscure online regions, cracks can ultimately appear in almost any system. In this climate, OSINT solutions are coming to the fore as essential tools for shedding light on dark net recesses, transforming investigations in different areas from law enforcement to corporate security. Here are some examples.

Combating Illegal Trade

As the foremost platform for conducting illicit trade, the Dark Web marketplaces are clearly the online areas where investigators need to be looking to find peddlers of all manner of contraband. However, with criminal actors taking care to conceal their activities behind layer upon layer of anonymization methods, you might wonder how OSINT tools can be used to overcome the Dark Web hurdles.

While PGP encryption is pretty infallible, human beings certainly are not. All it takes is a little slip for a fortified castle of anonymization to simply evaporate. The advanced, automated search methods of OSINT solutions are not only able to find such chinks in encrypted armour; they can also run expansive searches to link darknet activity to accounts on the Surface Web and identify users.

The deanonymization of the infamous darknet drugs marketplace Silk Road is a good example of how effective such techniques can be. On finding links between a PGP key, an email address, and a Reddit account, investigators appealed to the platform for user information such as log files and IP address. This data led to the identification of Silk Road’s chief administrator and the closure of the marketplace.

Countering Data Breaches

Leaked corporate and government data is a valuable commodity which enjoys widespread circulation on the Dark Web. According to CSO Spotlight, 60% of darknet content is potentially harmful to organizations. This stems from a range of threats including cyber espionage targeting classified data, critical infrastructure disruption caused by network attacks, and malware or ransomware deployment enabled by botnets.

Through their ability to continually monitor paste sites, discussion boards, and marketplaces across the Deep and Dark Web, OSINT tools enable security teams to promptly identify and contain data leaks, avoiding huge costs. Also, new forms of malware and attack strategy found on darknet sources can constantly inform threat intelligence for ever more accurate forecasting, mitigation, and investigation of future breaches.

Fighting Extremism

Posing significant threats to national security and society at large, extremism is a serious issue, which takes many forms and is often facilitated by the Dark Web. Dangerous radical sects such as Al-Qaeda and domestic far-right extremist groups harness the Dark Web’s anonymity to spread propaganda, seek financing, recruit new members, source weapons and even plan attacks.

Modern OSINT solutions possess a range of features allowing analysts to map out a truly comprehensive picture of a given group structure. By bringing together traced crypto routes, inter-domain account identification, geolocations, as well as various communications and interactions, investigators can identify and break up dangerous groups, while preempting and averting potential acts of terror.

Despite its reputation for anonymity and secrecy, the Dark Web is not nearly as obscure as people tend to believe. Through leveraging OSINT technologies, analysts and investigators from various sectors can shine a light on darknet activities and actors to gain crucial insights, transform cases, and accelerate work processes.