Open Source Intelligence (OSINT) is a set of activities, tools, and methods for obtaining and analyzing information from open sources. It applies to specific people, and organizations, as well as events, phenomena, and goals.
Table of Contents
Where and why OSINT is used
Open Source Intelligence (OSINT) as a separate discipline originated in the United States in the 1940s with the establishment of the Foreign Broadcast Information Service. Its employees recorded and analyzed short-wave radio transmissions of foreign countries, after which the received data was transmitted in the form of reports to military and intelligence agencies. Today, open-source intelligence is used not only in government security and defense agencies, but also in commercial companies, analytical agencies, political organizations, etc.
With Open Source Intelligence you can:
- receive the most objective and useful information for decision-making;
- obtain competitive advantages for your organization or its product;
- find flaws and vulnerabilities in your own security system, protect confidential information about customers;
- understand the psychological characteristics, needs, and habits of the target audience.
In the IT industry and information security, Open Source Intelligence helps:
- collect information about competitors and look for competitive advantages;
- analyze the security of the object, identify vulnerable points of the security system;
- find information leaks;
- identify possible threats, their sources, and direction;
- analyze cybercrimes (data theft, hacks, etc.).
OSINT can be used for both legal and illegal purposes. For example, using open-source intelligence, an attacker can:
- steal personal data of users or confidential information about the activities of a person/organization;
- obtain compromising evidence and use it for blackmail, extortion, damage to reputation, etc.;
- disrupt the work of information, production, or other resources of the organization.
It is necessary to distinguish between intelligence on open resources and the further use of the information obtained with its help.
Where do they get the data from
Open source intelligence involves obtaining data from sources in the public domain and/or those that can be accessed on demand. These include:
- information materials (articles, news, notes) in the media;
- scientific research published in specialized publications;
- books – encyclopedias, reference books, memoirs, etc.;
- posts and comments on social networks;
- information from the census;
- documents from open state and non-state archives;
- public commercial data (income, profit, loss, growth, share price, etc.);
- results of public polls;
- data from Earth remote sensing satellites and aerial photography aircraft;
- police and court documents and other sources.
What distinguishes OSINT from intelligence and espionage
Legality
The collection and analysis of information in the public domain does not contradict the norms of international law, as well as the laws of most states, although some sources and methods of their study may be on the verge of legality. During industrial or commercial espionage, illegal methods and tools are used to obtain information, such as bribery and blackmail of members of a competing organization, unauthorized entry into closed databases, theft of information constituting a trade secret, etc.
Availability
Any organization and even an individual can monitor and analyze publicly available sources without the use of specialized equipment or “connections” in the state security agencies.
Open Source Intelligence in the field of information security
With the development of the Internet, the focus of analysts has shifted to cyberspace as one of the main sources of information. Here, useful data can be:
- registration information about the certificate or site domain;
- open personal data of users (username, email addresses, phone numbers);
- user activity in social networks (posts, comments, etc.);
- user queries in search engines;
- Website HTML code;
- public text, graphic, audio, video files and their metadata (for example, date, time and place of creation, and device used);
- geolocation data and other types of information.
Much data can be accessed over the open Internet using resources indexed by search engines. However, sources from the Deep Web, to which ordinary users do not have access due to the need to pay for them, also fall under the definition of open source. In other words, OSINT works with all data that is not confidential, and does not constitute a commercial or state secret.
Open Source Intelligence Methods
All methods and tools used in open source intelligence can be divided into two categories.
Passive
Allows you to get general information about the object. It is collected manually or with the help of special services and tools that simplify the collection, systematization, and analysis of data. For example, programs for parsing sites. In fact, absolutely everyone who has a computer and Internet access can engage in passive web intelligence, from a simple user to an employee of an analytical or marketing department.
Passive methods include:
- collection of information (including from photographs) from open search engines;
- analysis of user activity in social networks and blogs, on forums, and other virtual platforms;
- search for open personal data of users in social networks, and instant messengers;
- viewing saved copies of sites in search engines, and Internet archives;
- obtaining geolocation data using public resources like Google Maps.
Active
Such methods imply the direct influence of the analyst on the object under study, the use of specialized means of obtaining data, or the performance of actions that require certain efforts, for example:
- collection of data on closed resources, access to which is possible only by subscription;
- the use of specialized services and programs that actively affect the object under study – for example, they are automatically registered on the site;
- use of services that scan applications, files, or websites for malicious code;
- creation of fake web resources, messenger channels that collect user data, and confidential or secret information.
In the OSINT logic, passive methods aimed at collecting general information from readily available sources precede the use of active methods designed to collect specific data about an object.
Tools for Open Source Intelligence (OSINT)
OSINT framework
It is the most comprehensive open-source data base available. They are grouped by category in an interactive map. By clicking on a particular class, you can go to a subclass, and in it – to a specific source of information. The OSINT framework does not promote any ideology, it is a purely informational resource aimed at simplifying Internet searches.
Shodan
This is a search engine designed to find devices connected to the Internet by IPv4 addresses (routers, CCTV cameras, security sensors, etc.). The system itself does no harm, but with its help, anyone with due diligence can find an unprotected or poorly protected device. It got its name from the antagonist of the System Shock games – a distraught artificial intelligence.
Metagoofil
It is a metasearch engine that uses other search engines to find and retrieve publicly available PDF, Word, Powerpoint, and Excel files. With its help, you can parse technical documentation, client databases, directories, catalogs, and other useful sources.
Open Source Intelligence is legal if the information obtained with its help is not used to the detriment of other people, organizations, or the state. Open-source intelligence has ceased to be the prerogative of the state security and defense agencies and is actively used in the civilian (primarily commercial) sphere.
