Skip to content

Commit ada9d68

Browse files
author
stlc-bot
committed
feat(scrape): add google-maps-reviews prebuilt parser with oldest-first sort (#1928)
Stainless-Generated-From: 53896a2d8f88944c2dbfd2ae3f71fa5ce5377564
1 parent 01b289c commit ada9d68

4 files changed

Lines changed: 96 additions & 24 deletions

File tree

‎src/context/dev/types/batch_get_results_response.py‎

Lines changed: 18 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -294,7 +294,7 @@ class DataOkResultMetadata(BaseModel):
294294

295295
class DataOkResultParsed(BaseModel):
296296
"""
297-
Custom rules return fields after content filters; unmatched fields are null ([] for lists). The google-maps parser returns business records in data.results, and the google-play parser returns one app record as data, google-play-reviews returns data.reviews, app-store and app-store-reviews do the same for the App Store, emails, phone-numbers, and links return data.emails, data.phone_numbers, and data.links from the rendered page; the store and Maps parsers read their data directly, independently of DOM filters and other outputs.
297+
Custom rules return fields after content filters; unmatched fields are null ([] for lists). The google-maps parser returns business records in data.results, google-maps-reviews returns data.reviews, and the google-play parser returns one app record as data, google-play-reviews returns data.reviews, app-store and app-store-reviews do the same for the App Store, emails, phone-numbers, and links return data.emails, data.phone_numbers, and data.links from the rendered page; the store and Maps parsers read their data directly, independently of DOM filters and other outputs.
298298
"""
299299

300300
data: Optional[Dict[str, object]] = None
@@ -307,6 +307,7 @@ class DataOkResultParsed(BaseModel):
307307
type: Literal[
308308
"custom",
309309
"google-maps",
310+
"google-maps-reviews",
310311
"google-play",
311312
"google-play-reviews",
312313
"app-store",
@@ -502,12 +503,13 @@ class DataOkResult(BaseModel):
502503
parsed: DataOkResultParsed
503504
"""
504505
Custom rules return fields after content filters; unmatched fields are null ([]
505-
for lists). The google-maps parser returns business records in data.results, and
506-
the google-play parser returns one app record as data, google-play-reviews
507-
returns data.reviews, app-store and app-store-reviews do the same for the App
508-
Store, emails, phone-numbers, and links return data.emails, data.phone_numbers,
509-
and data.links from the rendered page; the store and Maps parsers read their
510-
data directly, independently of DOM filters and other outputs.
506+
for lists). The google-maps parser returns business records in data.results,
507+
google-maps-reviews returns data.reviews, and the google-play parser returns one
508+
app record as data, google-play-reviews returns data.reviews, app-store and
509+
app-store-reviews do the same for the App Store, emails, phone-numbers, and
510+
links return data.emails, data.phone_numbers, and data.links from the rendered
511+
page; the store and Maps parsers read their data directly, independently of DOM
512+
filters and other outputs.
511513
"""
512514

513515
product: DataOkResultProduct
@@ -809,7 +811,7 @@ class DataErrorResultMetadata(BaseModel):
809811

810812
class DataErrorResultParsed(BaseModel):
811813
"""
812-
Custom rules return fields after content filters; unmatched fields are null ([] for lists). The google-maps parser returns business records in data.results, and the google-play parser returns one app record as data, google-play-reviews returns data.reviews, app-store and app-store-reviews do the same for the App Store, emails, phone-numbers, and links return data.emails, data.phone_numbers, and data.links from the rendered page; the store and Maps parsers read their data directly, independently of DOM filters and other outputs.
814+
Custom rules return fields after content filters; unmatched fields are null ([] for lists). The google-maps parser returns business records in data.results, google-maps-reviews returns data.reviews, and the google-play parser returns one app record as data, google-play-reviews returns data.reviews, app-store and app-store-reviews do the same for the App Store, emails, phone-numbers, and links return data.emails, data.phone_numbers, and data.links from the rendered page; the store and Maps parsers read their data directly, independently of DOM filters and other outputs.
813815
"""
814816

815817
data: Optional[Dict[str, object]] = None
@@ -822,6 +824,7 @@ class DataErrorResultParsed(BaseModel):
822824
type: Literal[
823825
"custom",
824826
"google-maps",
827+
"google-maps-reviews",
825828
"google-play",
826829
"google-play-reviews",
827830
"app-store",
@@ -1018,12 +1021,13 @@ class DataErrorResult(BaseModel):
10181021
parsed: DataErrorResultParsed
10191022
"""
10201023
Custom rules return fields after content filters; unmatched fields are null ([]
1021-
for lists). The google-maps parser returns business records in data.results, and
1022-
the google-play parser returns one app record as data, google-play-reviews
1023-
returns data.reviews, app-store and app-store-reviews do the same for the App
1024-
Store, emails, phone-numbers, and links return data.emails, data.phone_numbers,
1025-
and data.links from the rendered page; the store and Maps parsers read their
1026-
data directly, independently of DOM filters and other outputs.
1024+
for lists). The google-maps parser returns business records in data.results,
1025+
google-maps-reviews returns data.reviews, and the google-play parser returns one
1026+
app record as data, google-play-reviews returns data.reviews, app-store and
1027+
app-store-reviews do the same for the App Store, emails, phone-numbers, and
1028+
links return data.emails, data.phone_numbers, and data.links from the rendered
1029+
page; the store and Maps parsers read their data directly, independently of DOM
1030+
filters and other outputs.
10271031
"""
10281032

10291033
product: DataErrorResultProduct

‎src/context/dev/types/batch_submit_params.py‎

Lines changed: 46 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -242,6 +242,7 @@ class InputScrapeDataParseParamsPrebuiltParser(TypedDict, total=False):
242242
parser: Required[
243243
Literal[
244244
"google-maps",
245+
"google-maps-reviews",
245246
"google-play",
246247
"google-play-reviews",
247248
"app-store",
@@ -254,7 +255,15 @@ class InputScrapeDataParseParamsPrebuiltParser(TypedDict, total=False):
254255
"""google-maps parses Google Maps business data into data.results.
255256
256257
Accepts Maps place, search, and internal data URLs. Uses a US exit and English
257-
labels regardless of hl or Accept-Language. google-play parses a Play Store app
258+
labels regardless of hl or Accept-Language. google-maps-reviews returns the
259+
reviews of a place in data.reviews, in the same shape as the Serper reviews API
260+
(rating, date, isoDate, snippet, translatedSnippet, likes, user, media,
261+
response, link, id), plus language, guided ratings in details, and
262+
user.isLocalGuide; data also carries the place featureId, cid, and sort. sort
263+
and limit pick the order and count, and oldest is exact. It accepts Maps place,
264+
search, and ?cid= URLs, plus search.google.com/local/reviews URLs, so the
265+
google_maps_url and google_reviews_url fields of google-maps results work as
266+
input. An unknown place returns no reviews. google-play parses a Play Store app
258267
into data: details, developer contact, ratings, price, permissions, and data
259268
safety. google-play-reviews returns the 200 newest reviews of the app, with
260269
developer replies, in data.reviews. Both Play parsers accept app details URLs
@@ -279,6 +288,19 @@ class InputScrapeDataParseParamsPrebuiltParser(TypedDict, total=False):
279288
type: Required[Literal["prebuilt"]]
280289
"""Use a prebuilt parser instead of custom rules."""
281290

291+
limit: int
292+
"""google-maps-reviews only.
293+
294+
Maximum number of reviews to return. Defaults to 100; maximum 1000.
295+
"""
296+
297+
sort: Literal["newest", "oldest", "relevant", "highest", "lowest"]
298+
"""google-maps-reviews only.
299+
300+
Order of data.reviews. newest, relevant, highest, and lowest follow Google;
301+
oldest returns the earliest reviews first, which Google itself does not offer.
302+
"""
303+
282304

283305
InputScrapeDataParseParams: TypeAlias = Union[
284306
InputScrapeDataParseParamsCustomParser, InputScrapeDataParseParamsPrebuiltParser
@@ -785,6 +807,7 @@ class InputCrawlDataParseParamsPrebuiltParser(TypedDict, total=False):
785807
parser: Required[
786808
Literal[
787809
"google-maps",
810+
"google-maps-reviews",
788811
"google-play",
789812
"google-play-reviews",
790813
"app-store",
@@ -797,7 +820,15 @@ class InputCrawlDataParseParamsPrebuiltParser(TypedDict, total=False):
797820
"""google-maps parses Google Maps business data into data.results.
798821
799822
Accepts Maps place, search, and internal data URLs. Uses a US exit and English
800-
labels regardless of hl or Accept-Language. google-play parses a Play Store app
823+
labels regardless of hl or Accept-Language. google-maps-reviews returns the
824+
reviews of a place in data.reviews, in the same shape as the Serper reviews API
825+
(rating, date, isoDate, snippet, translatedSnippet, likes, user, media,
826+
response, link, id), plus language, guided ratings in details, and
827+
user.isLocalGuide; data also carries the place featureId, cid, and sort. sort
828+
and limit pick the order and count, and oldest is exact. It accepts Maps place,
829+
search, and ?cid= URLs, plus search.google.com/local/reviews URLs, so the
830+
google_maps_url and google_reviews_url fields of google-maps results work as
831+
input. An unknown place returns no reviews. google-play parses a Play Store app
801832
into data: details, developer contact, ratings, price, permissions, and data
802833
safety. google-play-reviews returns the 200 newest reviews of the app, with
803834
developer replies, in data.reviews. Both Play parsers accept app details URLs
@@ -822,6 +853,19 @@ class InputCrawlDataParseParamsPrebuiltParser(TypedDict, total=False):
822853
type: Required[Literal["prebuilt"]]
823854
"""Use a prebuilt parser instead of custom rules."""
824855

856+
limit: int
857+
"""google-maps-reviews only.
858+
859+
Maximum number of reviews to return. Defaults to 100; maximum 1000.
860+
"""
861+
862+
sort: Literal["newest", "oldest", "relevant", "highest", "lowest"]
863+
"""google-maps-reviews only.
864+
865+
Order of data.reviews. newest, relevant, highest, and lowest follow Google;
866+
oldest returns the earliest reviews first, which Google itself does not offer.
867+
"""
868+
825869

826870
InputCrawlDataParseParams: TypeAlias = Union[
827871
InputCrawlDataParseParamsCustomParser, InputCrawlDataParseParamsPrebuiltParser

‎src/context/dev/types/web_scrape_params.py‎

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -210,6 +210,7 @@ class ParseParamsPrebuiltParser(TypedDict, total=False):
210210
parser: Required[
211211
Literal[
212212
"google-maps",
213+
"google-maps-reviews",
213214
"google-play",
214215
"google-play-reviews",
215216
"app-store",
@@ -222,7 +223,15 @@ class ParseParamsPrebuiltParser(TypedDict, total=False):
222223
"""google-maps parses Google Maps business data into data.results.
223224
224225
Accepts Maps place, search, and internal data URLs. Uses a US exit and English
225-
labels regardless of hl or Accept-Language. google-play parses a Play Store app
226+
labels regardless of hl or Accept-Language. google-maps-reviews returns the
227+
reviews of a place in data.reviews, in the same shape as the Serper reviews API
228+
(rating, date, isoDate, snippet, translatedSnippet, likes, user, media,
229+
response, link, id), plus language, guided ratings in details, and
230+
user.isLocalGuide; data also carries the place featureId, cid, and sort. sort
231+
and limit pick the order and count, and oldest is exact. It accepts Maps place,
232+
search, and ?cid= URLs, plus search.google.com/local/reviews URLs, so the
233+
google_maps_url and google_reviews_url fields of google-maps results work as
234+
input. An unknown place returns no reviews. google-play parses a Play Store app
226235
into data: details, developer contact, ratings, price, permissions, and data
227236
safety. google-play-reviews returns the 200 newest reviews of the app, with
228237
developer replies, in data.reviews. Both Play parsers accept app details URLs
@@ -247,6 +256,19 @@ class ParseParamsPrebuiltParser(TypedDict, total=False):
247256
type: Required[Literal["prebuilt"]]
248257
"""Use a prebuilt parser instead of custom rules."""
249258

259+
limit: int
260+
"""google-maps-reviews only.
261+
262+
Maximum number of reviews to return. Defaults to 100; maximum 1000.
263+
"""
264+
265+
sort: Literal["newest", "oldest", "relevant", "highest", "lowest"]
266+
"""google-maps-reviews only.
267+
268+
Order of data.reviews. newest, relevant, highest, and lowest follow Google;
269+
oldest returns the earliest reviews first, which Google itself does not offer.
270+
"""
271+
250272

251273
ParseParams: TypeAlias = Union[ParseParamsCustomParser, ParseParamsPrebuiltParser]
252274

‎src/context/dev/types/web_scrape_response.py‎

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -267,7 +267,7 @@ class Metadata(BaseModel):
267267

268268
class Parsed(BaseModel):
269269
"""
270-
Custom rules return fields after content filters; unmatched fields are null ([] for lists). The google-maps parser returns business records in data.results, and the google-play parser returns one app record as data, google-play-reviews returns data.reviews, app-store and app-store-reviews do the same for the App Store, emails, phone-numbers, and links return data.emails, data.phone_numbers, and data.links from the rendered page; the store and Maps parsers read their data directly, independently of DOM filters and other outputs.
270+
Custom rules return fields after content filters; unmatched fields are null ([] for lists). The google-maps parser returns business records in data.results, google-maps-reviews returns data.reviews, and the google-play parser returns one app record as data, google-play-reviews returns data.reviews, app-store and app-store-reviews do the same for the App Store, emails, phone-numbers, and links return data.emails, data.phone_numbers, and data.links from the rendered page; the store and Maps parsers read their data directly, independently of DOM filters and other outputs.
271271
"""
272272

273273
data: Optional[Dict[str, object]] = None
@@ -280,6 +280,7 @@ class Parsed(BaseModel):
280280
type: Literal[
281281
"custom",
282282
"google-maps",
283+
"google-maps-reviews",
283284
"google-play",
284285
"google-play-reviews",
285286
"app-store",
@@ -471,12 +472,13 @@ class WebScrapeResponse(BaseModel):
471472
parsed: Parsed
472473
"""
473474
Custom rules return fields after content filters; unmatched fields are null ([]
474-
for lists). The google-maps parser returns business records in data.results, and
475-
the google-play parser returns one app record as data, google-play-reviews
476-
returns data.reviews, app-store and app-store-reviews do the same for the App
477-
Store, emails, phone-numbers, and links return data.emails, data.phone_numbers,
478-
and data.links from the rendered page; the store and Maps parsers read their
479-
data directly, independently of DOM filters and other outputs.
475+
for lists). The google-maps parser returns business records in data.results,
476+
google-maps-reviews returns data.reviews, and the google-play parser returns one
477+
app record as data, google-play-reviews returns data.reviews, app-store and
478+
app-store-reviews do the same for the App Store, emails, phone-numbers, and
479+
links return data.emails, data.phone_numbers, and data.links from the rendered
480+
page; the store and Maps parsers read their data directly, independently of DOM
481+
filters and other outputs.
480482
"""
481483

482484
product: Product

0 commit comments

Comments
 (0)