You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit ada9d68
Browse filesBrowse the repository at this point in the historyBrowse files
stlc-bot
committed
feat(scrape): add google-maps-reviews prebuilt parser with oldest-first sort (#1928)
Copy file name to clipboardExpand all lines: src/context/dev/types/batch_get_results_response.py
+18-14Lines changed: 18 additions & 14 deletions
Original file line number
Diff line number
Diff line change
@@ -294,7 +294,7 @@ class DataOkResultMetadata(BaseModel):
294
294
295
295
classDataOkResultParsed(BaseModel):
296
296
"""
297
-
Custom rules return fields after content filters; unmatched fields are null ([] for lists). The google-maps parser returns business records in data.results, and the google-play parser returns one app record as data, google-play-reviews returns data.reviews, app-store and app-store-reviews do the same for the App Store, emails, phone-numbers, and links return data.emails, data.phone_numbers, and data.links from the rendered page; the store and Maps parsers read their data directly, independently of DOM filters and other outputs.
297
+
Custom rules return fields after content filters; unmatched fields are null ([] for lists). The google-maps parser returns business records in data.results, google-maps-reviews returns data.reviews, and the google-play parser returns one app record as data, google-play-reviews returns data.reviews, app-store and app-store-reviews do the same for the App Store, emails, phone-numbers, and links return data.emails, data.phone_numbers, and data.links from the rendered page; the store and Maps parsers read their data directly, independently of DOM filters and other outputs.
298
298
"""
299
299
300
300
data: Optional[Dict[str, object]] =None
@@ -307,6 +307,7 @@ class DataOkResultParsed(BaseModel):
307
307
type: Literal[
308
308
"custom",
309
309
"google-maps",
310
+
"google-maps-reviews",
310
311
"google-play",
311
312
"google-play-reviews",
312
313
"app-store",
@@ -502,12 +503,13 @@ class DataOkResult(BaseModel):
502
503
parsed: DataOkResultParsed
503
504
"""
504
505
Custom rules return fields after content filters; unmatched fields are null ([]
505
-
for lists). The google-maps parser returns business records in data.results, and
506
-
the google-play parser returns one app record as data, google-play-reviews
507
-
returns data.reviews, app-store and app-store-reviews do the same for the App
508
-
Store, emails, phone-numbers, and links return data.emails, data.phone_numbers,
509
-
and data.links from the rendered page; the store and Maps parsers read their
510
-
data directly, independently of DOM filters and other outputs.
506
+
for lists). The google-maps parser returns business records in data.results,
507
+
google-maps-reviews returns data.reviews, and the google-play parser returns one
508
+
app record as data, google-play-reviews returns data.reviews, app-store and
509
+
app-store-reviews do the same for the App Store, emails, phone-numbers, and
510
+
links return data.emails, data.phone_numbers, and data.links from the rendered
511
+
page; the store and Maps parsers read their data directly, independently of DOM
512
+
filters and other outputs.
511
513
"""
512
514
513
515
product: DataOkResultProduct
@@ -809,7 +811,7 @@ class DataErrorResultMetadata(BaseModel):
809
811
810
812
classDataErrorResultParsed(BaseModel):
811
813
"""
812
-
Custom rules return fields after content filters; unmatched fields are null ([] for lists). The google-maps parser returns business records in data.results, and the google-play parser returns one app record as data, google-play-reviews returns data.reviews, app-store and app-store-reviews do the same for the App Store, emails, phone-numbers, and links return data.emails, data.phone_numbers, and data.links from the rendered page; the store and Maps parsers read their data directly, independently of DOM filters and other outputs.
814
+
Custom rules return fields after content filters; unmatched fields are null ([] for lists). The google-maps parser returns business records in data.results, google-maps-reviews returns data.reviews, and the google-play parser returns one app record as data, google-play-reviews returns data.reviews, app-store and app-store-reviews do the same for the App Store, emails, phone-numbers, and links return data.emails, data.phone_numbers, and data.links from the rendered page; the store and Maps parsers read their data directly, independently of DOM filters and other outputs.
813
815
"""
814
816
815
817
data: Optional[Dict[str, object]] =None
@@ -822,6 +824,7 @@ class DataErrorResultParsed(BaseModel):
822
824
type: Literal[
823
825
"custom",
824
826
"google-maps",
827
+
"google-maps-reviews",
825
828
"google-play",
826
829
"google-play-reviews",
827
830
"app-store",
@@ -1018,12 +1021,13 @@ class DataErrorResult(BaseModel):
1018
1021
parsed: DataErrorResultParsed
1019
1022
"""
1020
1023
Custom rules return fields after content filters; unmatched fields are null ([]
1021
-
for lists). The google-maps parser returns business records in data.results, and
1022
-
the google-play parser returns one app record as data, google-play-reviews
1023
-
returns data.reviews, app-store and app-store-reviews do the same for the App
1024
-
Store, emails, phone-numbers, and links return data.emails, data.phone_numbers,
1025
-
and data.links from the rendered page; the store and Maps parsers read their
1026
-
data directly, independently of DOM filters and other outputs.
1024
+
for lists). The google-maps parser returns business records in data.results,
1025
+
google-maps-reviews returns data.reviews, and the google-play parser returns one
1026
+
app record as data, google-play-reviews returns data.reviews, app-store and
1027
+
app-store-reviews do the same for the App Store, emails, phone-numbers, and
1028
+
links return data.emails, data.phone_numbers, and data.links from the rendered
1029
+
page; the store and Maps parsers read their data directly, independently of DOM
Copy file name to clipboardExpand all lines: src/context/dev/types/web_scrape_response.py
+9-7Lines changed: 9 additions & 7 deletions
Original file line number
Diff line number
Diff line change
@@ -267,7 +267,7 @@ class Metadata(BaseModel):
267
267
268
268
classParsed(BaseModel):
269
269
"""
270
-
Custom rules return fields after content filters; unmatched fields are null ([] for lists). The google-maps parser returns business records in data.results, and the google-play parser returns one app record as data, google-play-reviews returns data.reviews, app-store and app-store-reviews do the same for the App Store, emails, phone-numbers, and links return data.emails, data.phone_numbers, and data.links from the rendered page; the store and Maps parsers read their data directly, independently of DOM filters and other outputs.
270
+
Custom rules return fields after content filters; unmatched fields are null ([] for lists). The google-maps parser returns business records in data.results, google-maps-reviews returns data.reviews, and the google-play parser returns one app record as data, google-play-reviews returns data.reviews, app-store and app-store-reviews do the same for the App Store, emails, phone-numbers, and links return data.emails, data.phone_numbers, and data.links from the rendered page; the store and Maps parsers read their data directly, independently of DOM filters and other outputs.
271
271
"""
272
272
273
273
data: Optional[Dict[str, object]] =None
@@ -280,6 +280,7 @@ class Parsed(BaseModel):
280
280
type: Literal[
281
281
"custom",
282
282
"google-maps",
283
+
"google-maps-reviews",
283
284
"google-play",
284
285
"google-play-reviews",
285
286
"app-store",
@@ -471,12 +472,13 @@ class WebScrapeResponse(BaseModel):
471
472
parsed: Parsed
472
473
"""
473
474
Custom rules return fields after content filters; unmatched fields are null ([]
474
-
for lists). The google-maps parser returns business records in data.results, and
475
-
the google-play parser returns one app record as data, google-play-reviews
476
-
returns data.reviews, app-store and app-store-reviews do the same for the App
477
-
Store, emails, phone-numbers, and links return data.emails, data.phone_numbers,
478
-
and data.links from the rendered page; the store and Maps parsers read their
479
-
data directly, independently of DOM filters and other outputs.
475
+
for lists). The google-maps parser returns business records in data.results,
476
+
google-maps-reviews returns data.reviews, and the google-play parser returns one
477
+
app record as data, google-play-reviews returns data.reviews, app-store and
478
+
app-store-reviews do the same for the App Store, emails, phone-numbers, and
479
+
links return data.emails, data.phone_numbers, and data.links from the rendered
480
+
page; the store and Maps parsers read their data directly, independently of DOM
0 commit comments