|
4 | 4 | [](https://www.python.org/downloads/release/python-3100/)
|
5 | 5 | [](https://smithery.ai/server/@ScrapeGraphAI/scrapegraph-mcp)
|
6 | 6 |
|
7 |
| -A [Model Context Protocol](https://modelcontextprotocol.io/introduction) (MCP) server that provides access to the [ScapeGraph AI](https://scrapegraphai.com) API. It allows language models to use AI-powered web scraping capabilities. |
| 7 | +A production-ready [Model Context Protocol](https://modelcontextprotocol.io/introduction) (MCP) server that provides seamless integration with the [ScapeGraph AI](https://scrapegraphai.com) API. This server enables language models to leverage advanced AI-powered web scraping capabilities with enterprise-grade reliability. |
8 | 8 |
|
9 | 9 | ## Available Tools
|
10 | 10 |
|
11 |
| -The server exposes the following tools: |
| 11 | +The server provides the following enterprise-ready tools: |
12 | 12 |
|
13 |
| -- `markdownify(website_url: str)`: Convert any webpage into clean, formatted markdown |
14 |
| -- `smartscraper(user_prompt: str, website_url: str)`: Extract structured data from any webpage using AI |
15 |
| -- `searchscraper(user_prompt: str)`: Perform AI-powered web searches with structured results |
| 13 | +- `markdownify(website_url: str)`: Transform any webpage into clean, structured markdown format |
| 14 | +- `smartscraper(user_prompt: str, website_url: str)`: Leverage AI to extract structured data from any webpage |
| 15 | +- `searchscraper(user_prompt: str)`: Execute AI-powered web searches with structured, actionable results |
16 | 16 |
|
17 |
| -## Usage |
| 17 | +## Setup Instructions |
18 | 18 |
|
19 |
| -You'll need a ScapeGraph API key to use this server. You can obtain one by: |
| 19 | +To utilize this server, you'll need a ScapeGraph API key. Follow these steps to obtain one: |
20 | 20 |
|
21 |
| -1. Going to the [ScapeGraph Dashboard](https://dashboard.scrapegraphai.com) |
22 |
| -2. Creating an account and obtaining an API key |
| 21 | +1. Navigate to the [ScapeGraph Dashboard](https://dashboard.scrapegraphai.com) |
| 22 | +2. Create an account and generate your API key |
23 | 23 |
|
24 |
| -### Installing via Smithery |
| 24 | +### Automated Installation via Smithery |
25 | 25 |
|
26 |
| -To install ScrapeGraph API Integration Server for Claude Desktop automatically via [Smithery](https://smithery.ai/server/@ScrapeGraphAI/scrapegraph-mcp): |
| 26 | +For automated installation of the ScrapeGraph API Integration Server using [Smithery](https://smithery.ai/server/@ScrapeGraphAI/scrapegraph-mcp): |
27 | 27 |
|
28 | 28 | ```bash
|
29 | 29 | npx -y @smithery/cli install @ScrapeGraphAI/scrapegraph-mcp --client claude
|
30 | 30 | ```
|
31 | 31 |
|
32 |
| -### Claude for Desktop |
| 32 | +### Claude Desktop Configuration |
33 | 33 |
|
34 |
| -Update your `claude_desktop_config.json` (located in `~/Library/Application\ Support/Claude/claude_desktop_config.json` on macOS and `%APPDATA%/Claude/claude_desktop_config.json` on Windows) to include the following: |
| 34 | +Update your Claude Desktop configuration file with the following settings: |
35 | 35 |
|
36 | 36 | ```json
|
37 | 37 | {
|
38 | 38 | "mcpServers": {
|
39 |
| - "scrapegraph": { |
40 |
| - "command": "uvx", |
| 39 | + "@ScrapeGraphAI-scrapegraph-mcp": { |
| 40 | + "command": "npx", |
41 | 41 | "args": [
|
42 |
| - "scrapegraph_mcp" |
43 |
| - ], |
44 |
| - "env": { |
45 |
| - "SGAI_API_KEY": "YOUR_SCRAPEGRAPH_API_KEY" |
46 |
| - } |
| 42 | + "-y", |
| 43 | + "@smithery/cli@latest", |
| 44 | + "run", |
| 45 | + "@ScrapeGraphAI/scrapegraph-mcp", |
| 46 | + "--config", |
| 47 | + "\"{\\\"scrapegraphApiKey\\\":\\\"sgai-123456\\\"}\"" |
| 48 | + ] |
47 | 49 | }
|
48 | 50 | }
|
49 | 51 | }
|
50 | 52 | ```
|
51 | 53 |
|
52 |
| -## Example Queries |
| 54 | +The configuration file is located at: |
| 55 | +- Windows: `%APPDATA%/Claude/claude_desktop_config.json` |
| 56 | +- macOS: `~/Library/Application\ Support/Claude/claude_desktop_config.json` |
53 | 57 |
|
54 |
| -Once connected, you can ask Claude questions like: |
| 58 | +## Example Use Cases |
55 | 59 |
|
56 |
| -- "What are the main features of the ScapeGraph API?" |
57 |
| -- "Convert the ScapeGraph homepage into markdown" |
58 |
| -- "Extract the pricing information from the ScapeGraph website" |
59 |
| -- "Find information about the latest advancements in AI-powered web scraping" |
60 |
| -- "Summarize the content of the Python documentation website" |
| 60 | +The server enables sophisticated queries such as: |
| 61 | + |
| 62 | +- "Analyze and extract the main features of the ScapeGraph API" |
| 63 | +- "Generate a structured markdown version of the ScapeGraph homepage" |
| 64 | +- "Extract and analyze pricing information from the ScapeGraph website" |
| 65 | +- "Research and summarize recent developments in AI-powered web scraping" |
| 66 | +- "Create a comprehensive summary of the Python documentation website" |
61 | 67 |
|
62 | 68 | ## Error Handling
|
63 | 69 |
|
64 |
| -The server provides human-readable error messages for common issues: |
| 70 | +The server implements robust error handling with detailed, actionable error messages for: |
| 71 | + |
| 72 | +- API authentication issues |
| 73 | +- Malformed URL structures |
| 74 | +- Network connectivity failures |
| 75 | +- Rate limiting and quota management |
| 76 | + |
| 77 | +## Common Issues |
65 | 78 |
|
66 |
| -- API authentication errors |
67 |
| -- Invalid URL formats |
68 |
| -- Network connectivity problems |
| 79 | +### Windows-Specific Connection |
| 80 | + |
| 81 | +When running on Windows systems, you may need to use the following command to connect to the MCP server: |
| 82 | + |
| 83 | +```bash |
| 84 | +C:\Windows\System32\cmd.exe /c npx -y @smithery/cli@latest run @ScrapeGraphAI/scrapegraph-mcp --config "{\"scrapegraphApiKey\":\"sgai-123456\"}" |
| 85 | +``` |
| 86 | + |
| 87 | +This ensures proper execution in the Windows environment. |
69 | 88 |
|
70 | 89 | ## License
|
71 | 90 |
|
72 |
| -This project is licensed under the MIT License - see the LICENSE file for details. |
| 91 | +This project is distributed under the MIT License. For detailed terms and conditions, please refer to the LICENSE file. |
| 92 | + |
| 93 | +Made with ❤️ by ScrapeGraphAI Team |
0 commit comments