Replies: 10 comments
|
2025-09-11 21:55:18,044 [INFO ] Starting transfer |
|
Laptop, local dockerized postgres |
|
commit: 343af2b1e2654d4fdfdce345fa9edb9d01f6ee7b. local This is significantly slower than previous versions because of the state, county, and quad_name lookup. |
|
@jacob-a-brown removed state,county,quadname lookup. run cleanup_wells as a separate process 2025-09-18 07:14:16,229 [INFO ] ********** STARTING TRANSFER ********** |
|
It certainly makes it slower, and often times out, but shouldn't it still be there for data fidelity and integrity? We could also retrieve those values with GET requests and not store them, but doing so would make the requests take way longer and render them unusable with the frontend. |
|
I want the state, county, and quad values set during a separate process e.g. |
|
|
Cloud run job limit=100 |
|
Cloud run job limit=500 |
|
Cloud run job limit=5000 |
Uh oh!
There was an error while loading. Please reload this page.
Laptop, cloud-sql-proxy
********** TRANSFERRING WELLS **********
Processing row 25 of 28411. NM-11224, avg rows per second: 0.39
Processing row 50 of 28411. NM-21466, avg rows per second: 0.42
Processing row 75 of 28411. NM-21486, avg rows per second: 0.42
All reactions