Skip to content

realstorypro/agent

Folders and files

NameName
Last commit message
Last commit date

Latest commit

author
Leonid Medovyy
Mar 6, 2022
71ad6c1 Â· Mar 6, 2022

History

63 Commits
Mar 6, 2022
Jan 24, 2022
Feb 5, 2022
Feb 5, 2022
Mar 6, 2022
Jan 24, 2022
Feb 1, 2022
Jan 24, 2022
Jan 24, 2022
Jan 24, 2022
Jan 24, 2022
Jan 24, 2022
Mar 2, 2022
Jan 26, 2022
Jan 26, 2022
Jan 24, 2022
Feb 5, 2022
Feb 16, 2022
Mar 6, 2022
Jan 26, 2022
Jan 26, 2022

Repository files navigation

The Agent 🥷

"Nobody can give you freedom. Nobody can give you equality or justice or anything. If you're a man, you take it." — Malcom X

Features

  • Companies Prospecting
  • Distributed Crunchbase Company Scrape
  • Contact Enrichment
  • Customer.Io Upload

Setup

  1. Setup mitmproxy
brew install mitmproxy
  1. Start the mitmproxy to disable peremetrix
 mitmproxy -s mitmproxy/crunchbase.py  
  1. Setup ENV file (including AGENT_CODENAME)

Usage

Prospecting Crunchbase

  1. Build a list
rake list:build
  1. Upload the list
rake list:upload

Scraping Crunchbase

  1. Take companies to scrape (550 Max)
rake agent:take
  1. Scrape with the following command
rake crunchbase:scrape
  1. Stop the scraper with
kill $(ps aux | grep '[s]crape' | awk '{print $2}')

Uploading to Customer.io

Note: This uploads all contacts not just the ones assigned to the agent. It's best to run this after 5pm (MST) so that we can run enchance right after

rake contacts:process

Enrich contacts w/ Timezone info

rake contacts:enrich 

About

🥷 stealth scraping with style

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 2

Languages