@@ -41,27 +41,30 @@ yarn add scrapegraph-js
41
41
42
42
## Quick Start
43
43
44
- Initialize with your API key:
44
+ ### Basic example
45
+
46
+ <Note >
47
+ Store your API keys securely in environment variables. Use ` .env ` files and libraries like ` dotenv ` to load them into your app.
48
+ </Note >
45
49
46
50
``` javascript
47
51
import { smartScraper } from ' scrapegraph-js' ;
52
+ import ' dotenv/config' ;
48
53
49
- const apiKey = process .env .SGAI_APIKEY ;
54
+ // Initialize variables
55
+ const apiKey = process .env .SGAI_APIKEY ; // Set your API key as an environment variable
50
56
const websiteUrl = ' https://example.com' ;
51
- const prompt = ' Extract the main heading and description ' ;
57
+ const prompt = ' What does the company do? ' ;
52
58
53
59
try {
54
- const response = await smartScraper (apiKey, websiteUrl, prompt);
60
+ const response = await smartScraper (apiKey, websiteUrl, prompt); // call SmartScraper function
55
61
console .log (response .result );
56
62
} catch (error) {
57
63
console .error (' Error:' , error);
58
64
}
65
+ ;
59
66
```
60
67
61
- <Note >
62
- Store your API keys securely in environment variables. Use ` .env ` files and libraries like ` dotenv ` to load them into your app.
63
- </Note >
64
-
65
68
## Services
66
69
67
70
### SmartScraper
@@ -76,10 +79,19 @@ const response = await smartScraper(
76
79
);
77
80
```
78
81
82
+ #### Parameters
83
+
84
+ | Parameter | Type | Required | Description |
85
+ | -----------| ------| ----------| -------------|
86
+ | apiKey | string | Yes | The ScrapeGraph API Key. |
87
+ | websiteUrl | string | Yes | The URL of the webpage that needs to be scraped. |
88
+ | prompt | string | Yes | A textual description of what you want to achieve. |
89
+ | schema | object | No | The Pydantic or Zod object that describes the structure and format of the response. |
90
+
79
91
<Accordion title = " Basic Schema Example" icon = " code" >
80
92
Define a simple schema using Zod:
81
93
82
- ``` typescript
94
+ ``` javascript
83
95
import { z } from ' zod' ;
84
96
85
97
const ArticleSchema = z .object ({
@@ -108,7 +120,7 @@ console.log(`Published: ${response.result.publishDate}`);
108
120
<Accordion title = " Advanced Schema Example" icon = " code" >
109
121
Define a complex schema for nested data structures:
110
122
111
- ``` typescript
123
+ ``` javascript
112
124
import { z } from ' zod' ;
113
125
114
126
const EmployeeSchema = z .object ({
@@ -169,10 +181,18 @@ const response = await searchScraper(
169
181
);
170
182
```
171
183
184
+ #### Parameters
185
+
186
+ | Parameter | Type | Required | Description |
187
+ | -----------| ------| ----------| -------------|
188
+ | apiKey | string | Yes | The ScrapeGraph API Key. |
189
+ | prompt | string | Yes | A textual description of what you want to achieve. |
190
+ | schema | object | No | The Pydantic or Zod object that describes the structure and format of the response |
191
+
172
192
<Accordion title = " Basic Schema Example" icon = " code" >
173
193
Define a simple schema using Zod:
174
194
175
- ``` typescript
195
+ ``` javascript
176
196
import { z } from ' zod' ;
177
197
178
198
const ArticleSchema = z .object ({
@@ -199,7 +219,7 @@ console.log(`Published: ${response.result.publishDate}`);
199
219
<Accordion title = " Advanced Schema Example" icon = " code" >
200
220
Define a complex schema for nested data structures:
201
221
202
- ``` typescript
222
+ ``` javascript
203
223
import { z } from ' zod' ;
204
224
205
225
const EmployeeSchema = z .object ({
@@ -230,19 +250,6 @@ const response = await searchScraper(
230
250
' Find the best restaurants in San Francisco' ,
231
251
RestaurantSchema
232
252
);
233
-
234
-
235
-
236
- // Access nested data
237
- console .log (` Restaurant: ${response .result .name } ` );
238
- console .log (' \n Address:' );
239
- response .result .address .forEach (address => {
240
- console .log (` - ${address } ` );
241
- });
242
-
243
-
244
- console .log (' \n Rating:' );
245
- console .log (` - ${response .result .rating } ` );
246
253
```
247
254
</Accordion >
248
255
@@ -258,6 +265,12 @@ const response = await markdownify(
258
265
' https://example.com'
259
266
);
260
267
```
268
+ #### Parameters
269
+
270
+ | Parameter | Type | Required | Description |
271
+ | -----------| ------| ----------| -------------|
272
+ | apiKey | string | Yes | The ScrapeGraph API Key. |
273
+ | websiteUrl | string | Yes | The URL of the webpage to convert to markdown. |
261
274
262
275
## API Credits
263
276
0 commit comments