Skip to content

Commit f54ba9d

Browse files
author
stlc-bot
committed
feat(scrape): add Google Play app and reviews prebuilt parsers (#1857)
feat(scrape): add App Store app and reviews prebuilt parsers (#1859) feat(scrape): add emails prebuilt parser (#1862) feat(scrape): add phone-numbers prebuilt parser (#1863) feat(scrape): add links prebuilt parser (#1865) Stainless-Generated-From: 511234daf70222effdd906da90775ef07fb30693
1 parent c8faea0 commit f54ba9d

2 files changed

Lines changed: 51 additions & 8 deletions

File tree

‎src/context/dev/types/web_scrape_params.py‎

Lines changed: 33 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -207,12 +207,41 @@ class ParseParamsCustomParser(TypedDict, total=False):
207207

208208

209209
class ParseParamsPrebuiltParser(TypedDict, total=False):
210-
parser: Required[Literal["google-maps"]]
211-
"""Parse Google Maps business data into data.results.
210+
parser: Required[
211+
Literal[
212+
"google-maps",
213+
"google-play",
214+
"google-play-reviews",
215+
"app-store",
216+
"app-store-reviews",
217+
"emails",
218+
"phone-numbers",
219+
"links",
220+
]
221+
]
222+
"""google-maps parses Google Maps business data into data.results.
212223
213224
Accepts Maps place, search, and internal data URLs. Uses a US exit and English
214-
labels regardless of hl or Accept-Language, and reads Maps data independently of
215-
DOM filters.
225+
labels regardless of hl or Accept-Language. google-play parses a Play Store app
226+
into data: details, developer contact, ratings, price, permissions, and data
227+
safety. google-play-reviews returns the 200 newest reviews of the app, with
228+
developer replies, in data.reviews. Both Play parsers accept app details URLs
229+
(play.google.com/store/apps/details?id=…); hl and gl set the language and
230+
storefront, defaulting to en and us. app-store parses an App Store app into
231+
data: details, developer, ratings, price, version, and screenshots.
232+
app-store-reviews returns up to the 500 newest reviews in data.reviews. Both App
233+
Store parsers accept app URLs (apps.apple.com/us/app/…/id…); the country in the
234+
path sets the storefront, defaulting to us. emails accepts any page and returns
235+
its email addresses as data.emails ([{ email }]), read from mailto links,
236+
Cloudflare-protected addresses, and visible text. phone-numbers accepts any page
237+
and returns its valid phone numbers in E.164 as data.phone_numbers ([{ phone
238+
}]), read from tel links and visible text; numbers without a country code are
239+
read for the country of the site TLD, defaulting to US. links accepts any page
240+
and returns its unique http(s) links as data.links ([{ url, text }]), with
241+
absolute URLs; text falls back to aria-label or image alt and is null when a
242+
link has none. The emails parser reads the rendered page and respects content
243+
filters; the other prebuilt parsers read their data independently of DOM
244+
filters.
216245
"""
217246

218247
type: Required[Literal["prebuilt"]]

‎src/context/dev/types/web_scrape_response.py‎

Lines changed: 18 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -267,7 +267,7 @@ class Metadata(BaseModel):
267267

268268
class Parsed(BaseModel):
269269
"""
270-
Custom rules return fields after content filters; unmatched fields are null ([] for lists). The google-maps parser returns business records in data.results and reads Maps data directly, independently of DOM filters and other outputs.
270+
Custom rules return fields after content filters; unmatched fields are null ([] for lists). The google-maps parser returns business records in data.results, and the google-play parser returns one app record as data, google-play-reviews returns data.reviews, app-store and app-store-reviews do the same for the App Store, emails, phone-numbers, and links return data.emails, data.phone_numbers, and data.links from the rendered page; the store and Maps parsers read their data directly, independently of DOM filters and other outputs.
271271
"""
272272

273273
data: Optional[Dict[str, object]] = None
@@ -277,7 +277,17 @@ class Parsed(BaseModel):
277277
success: Optional[bool] = None
278278
"""`true` if returned, `false` if it failed, `null` if not requested."""
279279

280-
type: Literal["custom", "google-maps"]
280+
type: Literal[
281+
"custom",
282+
"google-maps",
283+
"google-play",
284+
"google-play-reviews",
285+
"app-store",
286+
"app-store-reviews",
287+
"emails",
288+
"phone-numbers",
289+
"links",
290+
]
281291
"""
282292
The selected prebuilt parser name, or custom when type is omitted, custom rules
283293
are used, or parsing is not requested.
@@ -461,8 +471,12 @@ class WebScrapeResponse(BaseModel):
461471
parsed: Parsed
462472
"""
463473
Custom rules return fields after content filters; unmatched fields are null ([]
464-
for lists). The google-maps parser returns business records in data.results and
465-
reads Maps data directly, independently of DOM filters and other outputs.
474+
for lists). The google-maps parser returns business records in data.results, and
475+
the google-play parser returns one app record as data, google-play-reviews
476+
returns data.reviews, app-store and app-store-reviews do the same for the App
477+
Store, emails, phone-numbers, and links return data.emails, data.phone_numbers,
478+
and data.links from the rendered page; the store and Maps parsers read their
479+
data directly, independently of DOM filters and other outputs.
466480
"""
467481

468482
product: Product

0 commit comments

Comments
 (0)