Skip to content

Home

Jump to bottom
YogSec edited this page Feb 20, 2025 · 1 revision

YogSec Search - Wiki

Introduction

YogSec Search is a specialized search tool that leverages Bing search results to simplify data gathering for researchers, developers, and cybersecurity professionals. It automates the retrieval of search results while filtering out irrelevant tracking URLs and eliminating duplicates. This tool aims to enhance productivity and precision when gathering online information.

Problem Statement

Manual search engine queries are often inefficient and time-consuming, especially when gathering a large number of search results. Users may encounter:

  • Irrelevant tracking URLs (e.g., bing.com/ck/a).
  • Duplicated results.
  • Limited control over the number of search results.

These issues hinder productivity, making it difficult to quickly obtain clean and reliable search data.

Solution

YogSec Search addresses these challenges by automating Bing search queries with the following capabilities:

  • Scrapes search results directly from Bing.
  • Removes tracking URLs and unwanted Bing-generated redirect links.
  • Filters duplicate URLs from the output.
  • Allows fetching up to 200 search results.
  • Supports saving both titles with URLs and URLs only to files.

Target Users

YogSec Search is useful for:

  • Security Researchers: Collect target URLs for reconnaissance and vulnerability assessments.
  • Bug Bounty Hunters: Streamline information gathering during bug bounty hunting.
  • Developers: Automate search queries to gather research data.
  • Investigative Journalists: Fetch clean search data without cluttered results.
  • Digital Marketers: Collect competitor website information for market analysis.

Benefits

  • Time Efficiency: Automates search processes, eliminating manual result extraction.
  • Clean Data: Provides clean URLs without Bing redirections.
  • Customizability: Allows users to specify the number of results and customize output.

Installation

Prerequisites

Ensure you have Python 3.x installed.

Installation Steps

git clone https://github.com/YogSec/YogSec-Search.git
cd YogSec-Search
pip install -r requirements.txt

Functionalities

Feature Description
Search Automation Automates search queries to Bing.
Tracking URL Removal Filters out tracking URLs from Bing (e.g., bing.com/ck/a).
Duplicate Removal Ensures unique URLs in the output.
Custom Result Count Fetches up to 200 search results with -i flag.
Save Full Results Saves titles with URLs using -s option.
Save URLs Only Saves URLs only using -su option.
Help & Version Info Displays help message and version details.

Examples

1. Basic Search

python search.py "site:example.com"

2. Fetch 200 Results

python search.py -i 200 "site:example.com"

3. Save Results to a File

python search.py -s results.txt "site:example.com"

4. Save URLs Only to a File

python search.py -su urls.txt "site:example.com"

5. Combine Options

python search.py -i 200 -s results.txt -su urls.txt "site:example.com"

Customization

The script allows user agent rotation for better evasion. You can edit the USER_AGENTS list in search.py to add more user agents.

Dependencies

  • requests
  • BeautifulSoup4

Install them with:

pip install requests beautifulsoup4

Version

Current Version: 1.0

License

Licensed under the MIT License.

Author

Developed by Abhinav Singwal | YogSec

Contact:

# YogSec Search - Wiki

Introduction

YogSec Search is a specialized search tool that leverages Bing search results to simplify data gathering for researchers, developers, and cybersecurity professionals. It automates the retrieval of search results while filtering out irrelevant tracking URLs and eliminating duplicates. This tool aims to enhance productivity and precision when gathering online information.

Problem Statement

Manual search engine queries are often inefficient and time-consuming, especially when gathering a large number of search results. Users may encounter:

  • Irrelevant tracking URLs (e.g., bing.com/ck/a).
  • Duplicated results.
  • Limited control over the number of search results.

These issues hinder productivity, making it difficult to quickly obtain clean and reliable search data.

Solution

YogSec Search addresses these challenges by automating Bing search queries with the following capabilities:

  • Scrapes search results directly from Bing.
  • Removes tracking URLs and unwanted Bing-generated redirect links.
  • Filters duplicate URLs from the output.
  • Allows fetching up to 200 search results.
  • Supports saving both titles with URLs and URLs only to files.

Target Users

YogSec Search is useful for:

  • Security Researchers: Collect target URLs for reconnaissance and vulnerability assessments.
  • Bug Bounty Hunters: Streamline information gathering during bug bounty hunting.
  • Developers: Automate search queries to gather research data.
  • Investigative Journalists: Fetch clean search data without cluttered results.
  • Digital Marketers: Collect competitor website information for market analysis.

Benefits

  • Time Efficiency: Automates search processes, eliminating manual result extraction.
  • Clean Data: Provides clean URLs without Bing redirections.
  • Customizability: Allows users to specify the number of results and customize output.

Installation

Prerequisites

Ensure you have Python 3.x installed.

Installation Steps

git clone https://github.com/YogSec/YogSec-Search.git
cd YogSec-Search
pip install bs4

Functionalities

Feature Description
Search Automation Automates search queries to Bing.
Tracking URL Removal Filters out tracking URLs from Bing (e.g., bing.com/ck/a).
Duplicate Removal Ensures unique URLs in the output.
Custom Result Count Fetches up to 200 search results with -i flag.
Save Full Results Saves titles with URLs using -s option.
Save URLs Only Saves URLs only using -su option.
Help & Version Info Displays help message and version details.

Usage

python search.py [OPTIONS] QUERY

Options

Option Description
-h Display help message.
-v Show version information.
-i [NUM] Number of search results (default: 100, max: 200).
-s [FILENAME] Save title + URL output to a file.
-su [FILENAME] Save only URLs to a file.

Examples

1. Basic Search

python search.py "site:example.com"

2. Fetch 200 Results

python search.py -i 200 "site:example.com"

3. Save Results to a File

python search.py -s results.txt "site:example.com"

4. Save URLs Only to a File

python search.py -su urls.txt "site:example.com"

5. Combine Options

python search.py -i 200 -s results.txt -su urls.txt "site:example.com"

Customization

The script allows user agent rotation for better evasion. You can edit the USER_AGENTS list in search.py to add more user agents.

Dependencies

  • requests
  • BeautifulSoup4

Install them with:

pip install requests bs4

Version

Current Version: 1.0

License

Licensed under the MIT License.

Author

Developed by Abhinav Singwal | YogSec

Contact:

Clone this wiki locally