Skip to content

Commit fbc48da

Browse files
authored
feat: add content migrate-to-connect-cloud (#838)
* feat: add support for deploying to Posit Connect Cloud Adds Posit Connect Cloud as a deployment target alongside Posit Connect and shinyapps.io, mirroring the R rsconnect package's support: - Select the target with --connect-cloud or -s connect.posit.cloud. - Authenticate with an interactive OAuth device-code login or a service account client ID/secret (client credentials grant), with automatic token refresh and write-back to the credential store. - Register credentials with `rsconnect add`, verifying the account exists and grants the content:create permission before storing. - Deploy through the Connect Cloud revision model: create or update content, upload the bundle to a presigned URL, publish, poll the revision, and print the publish log from the logs service on failure. - Record deployments locally before publishing, since Connect Cloud cannot look content up by name. - Support the production, staging, and development environments via CONNECT_CLOUD_ENVIRONMENT, pinned to the saved server's URL. Fixes #817 * fix: repair Connect Cloud log authorization on Python 3.8 `cast(dict[str, Any], ...)` evaluates its first argument at runtime, so `from __future__ import annotations` does not cover it and `dict[...]` raises TypeError on Python 3.8. Describe the response with a TypedDict instead, matching the other ConnectCloudClient methods. * feat: add `content migrate-to-connect-cloud` Point a directory's local deployment record at an existing Posit Connect Cloud content item, so the next deploy from that directory updates that item instead of creating a second one. Connect Cloud cannot look content up by name, so the local record is the only way back to a content item; without one, deploying content that already exists there duplicates it. Nothing is copied and no bundle is uploaded. The record that was migrated from is removed, leaving the directory with one deployment target rather than two; `--from-server` chooses which record to migrate when the store covers several servers, and `-o/--overwrite` replaces an existing Connect Cloud record. Records are keyed by account, and a deploy refuses content owned by another account, so a record written under the wrong account would be silently useless. Migration therefore requires the content's owning account to be the one being targeted and names it when it is not. With no local record at all the record is reconstructed from the content, with app mode `unknown`, which `validate_app_mode` already tolerates. Fixes #826 * feat: add `content migrate-to-connect-cloud` Point a directory's local deployment record at an existing Posit Connect Cloud content item, so the next deploy from that directory updates that item instead of creating a second one. Connect Cloud cannot look content up by name, so the local record is the only way back to a content item; without one, deploying content that already exists there duplicates it. Nothing is copied and no bundle is uploaded. The record that was migrated from is removed, leaving the directory with one deployment target rather than two; `--from-server` chooses which record to migrate when the store covers several servers, and `-o/--overwrite` replaces an existing Connect Cloud record. Records are keyed by account, and a deploy refuses content owned by another account, so a record written under the wrong account would be silently useless. Migration therefore requires the content's owning account to be the one being targeted and names it when it is not. With no local record at all the record is reconstructed from the content, with app mode `unknown`, which `validate_app_mode` already tolerates. Widening `AppMetadata.app_guid` to `Optional[str]` also types the None that the Connect Cloud deploy in #840 already passes. Fixes #826 * fix: refuse unusable Connect Cloud migration targets Two ways `content migrate-to-connect-cloud` could remove the source deployment record and leave behind one that cannot be deployed. A viewer role on the owning account passed the ownership check. `validate_connect_cloud_server` resolves publish permission through `get_account_by_name` only when the account id is not already known, so an id saved with a nickname skipped it and the refusal came from the deploy instead, after the source record was gone. The account is now checked before anything is written, reusing `_can_publish` so a viewer role is judged the same way in both places. The id-keyed and name-keyed record locations collide when an account's name equals its id, and removing the name-keyed record then deleted the record just written, emptying the store and reporting "The deployment record could not be saved." The fallback is now removed only when it differs from the target. Also drop the claim that migration leaves one deployment target: with `--from-server`, records for other servers are deliberately kept. * fix: keep a Posit Connect record when migrating to Connect Cloud A lone deployment record for a Posit Connect server was silently taken as the migration source and removed, so the next deploy to that server created new content instead of updating the existing item — the duplication this command exists to prevent, on the Connect side. Removing the source is only justified for shinyapps.io: that content has been migrated away, so its record is dead. Connect content still exists and is still deployable, and deploying one directory to both Connect and Connect Cloud is a supported setup. The source record still supplies the title and app mode either way; only a shinyapps.io record is removed. `--from-server` chooses which record supplies the metadata; it does not make a live Connect record removable. Reported by vrsarah in review of #838. * fix: match --from-server against a record URL with a trailing slash Deployment record keys are the server URL as typed at deploy time -- AbstractRemoteServer stores it verbatim -- so a record can be keyed "https://connect.example.com/" while --from-server names it without the slash, or the reverse. Either way migration_source_record's exact == found no match and the migration was refused. The rest of the function was already slash-insensitive: is_connect_cloud_url tolerates a trailing slash, and _is_shinyapps_record strips one. A shinyapps record stored with a slash was therefore classified correctly, and would have been removed by the migration, but could not be selected. Route all three comparisons through one _record_server_url helper instead of spelling the expression out per site. --from-server is stripped before resolve_server_alias, since the shinyapps.io short name resolves by exact comparison. * feat: accept a saved nickname for --from-server --from-server resolved only the two hard-coded pseudo server names (shinyapps.io, connect.posit.cloud) and never consulted the server store, so a Posit Connect instance had to be named by URL while shinyapps.io could be named by name. Look the value up as a nickname first, and fall back to the URL and alias handling. A nickname for a saved shinyapps.io server resolves to the same API URL as "shinyapps.io", so its record is removed like any other shinyapps.io source. A Connect record is still kept, as bd7cd7b established. Tested both ways. The option help called the value a URL while giving `shinyapps.io` as the example; it now names both forms. TestConnectCloudMigrate gets a temp-dir ServerStore, since resolving a nickname would otherwise read the developer's own saved servers.
1 parent 055d9fd commit fbc48da

5 files changed

Lines changed: 870 additions & 7 deletions

File tree

‎docs/CHANGELOG.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
99

1010
- Posit Connect Cloud is now a supported deployment target, alongside Posit
1111
Connect and shinyapps.io.
12+
- New `rsconnect content migrate-to-connect-cloud` points a directory's local
13+
deployment record at an existing Posit Connect Cloud content item, so the next
14+
deploy from that directory updates that item instead of creating a second one.
15+
Use it after migrating shinyapps.io content to Connect Cloud:
16+
`rsconnect content migrate-to-connect-cloud ./my-app -n cloud
17+
--content-id <id>`. Nothing is copied and no bundle is uploaded — the content
18+
must already exist in Connect Cloud, and only local files change. A
19+
shinyapps.io record is removed once its content has been migrated, since that
20+
record is then dead; a record for a Posit Connect server is kept, so the
21+
directory can go on deploying to both. Pass `--from-server` to choose which
22+
record to migrate when there are several.
1223
- `rsconnect add` now reports invalid option combinations and unreadable
1324
certificate files as plain error messages. Previously these surfaced as raw
1425
Python tracebacks.

‎rsconnect/api.py‎

Lines changed: 240 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,15 @@
7070
create_multipart_form_data,
7171
)
7272
from .log import cls_logged, connect_logger, console_logger, logger
73-
from .metadata import SHINYAPPS_API_URL, SHINYAPPS_SERVER_NAME, AppStore, ServerData, ServerStore
73+
from .metadata import (
74+
SHINYAPPS_API_URL,
75+
SHINYAPPS_SERVER_NAME,
76+
AppMetadata,
77+
AppStore,
78+
ServerData,
79+
ServerStore,
80+
resolve_server_alias,
81+
)
7482
from .models import (
7583
AppMode,
7684
AppModes,
@@ -1273,6 +1281,49 @@ class ServerDetails(TypedDict):
12731281
python: ServerDetailsPython
12741282

12751283

1284+
def _record_server_list(records: list[AppMetadata]) -> str:
1285+
"""The servers a set of deployment records covers, for an error message."""
1286+
servers = sorted(record.get("server_url", "") for record in records)
1287+
if not servers:
1288+
return ""
1289+
return " Records exist for: %s." % ", ".join(servers)
1290+
1291+
1292+
def _record_server_url(record: AppMetadata) -> str:
1293+
"""The server a record's key names, without a Cloud account or trailing slash."""
1294+
return record.get("server_url", "").split("#")[0].rstrip("/")
1295+
1296+
1297+
def _names_a_server(value: str) -> bool:
1298+
"""Whether a --from-server value identifies a server on its own.
1299+
1300+
A URL does, and so does a pseudo server name the alias table resolves -- in
1301+
either spelling, since a trailing slash is accepted. Nothing validates
1302+
nicknames, so one spelled like any of those must not shadow it.
1303+
"""
1304+
stripped = value.rstrip("/")
1305+
return "://" in value or resolve_server_alias(value) != value or resolve_server_alias(stripped) != stripped
1306+
1307+
1308+
def _migration_source_keys(from_server: str) -> tuple[str, str]:
1309+
"""The record key a --from-server value names, as given and slash-insensitive.
1310+
1311+
A saved nickname names a server as well as a URL does, and records are keyed by
1312+
the URL, so the store has to be consulted for a bare name.
1313+
"""
1314+
named = resolve_server_alias(from_server)
1315+
if not _names_a_server(from_server):
1316+
entry = ServerStore().get_by_name(from_server)
1317+
if entry:
1318+
named = entry["url"]
1319+
return named, resolve_server_alias(named.rstrip("/")).rstrip("/")
1320+
1321+
1322+
def _is_shinyapps_record(record: AppMetadata) -> bool:
1323+
"""Whether a deployment record points at shinyapps.io."""
1324+
return _record_server_url(record) == SHINYAPPS_API_URL
1325+
1326+
12761327
class RSConnectExecutor:
12771328
def __init__(
12781329
self,
@@ -2236,6 +2287,167 @@ def write_deployed_info(self):
22362287
self.app_mode,
22372288
)
22382289

2290+
def migration_source_record(self, from_server: Optional[str] = None) -> Optional[AppMetadata]:
2291+
"""The deployment record being migrated away from, if there is one.
2292+
2293+
Supplies the title and app mode for the new record, and is removed only
2294+
when it is a shinyapps.io record (see migrate_to_connect_cloud).
2295+
2296+
Only records for other servers are candidates: a Connect Cloud record is
2297+
what this migration produces, so treating one as a source would delete the
2298+
result. With no `from_server` a lone record is taken, and several are
2299+
reported rather than picked between.
2300+
"""
2301+
candidates = [
2302+
record
2303+
for record in self.app_store.get_all()
2304+
if not connect_cloud.is_connect_cloud_url(_record_server_url(record))
2305+
]
2306+
2307+
if from_server:
2308+
named, target = _migration_source_keys(from_server)
2309+
# An exact key wins: records can exist for both slash variants of one URL,
2310+
# and then only the spelling the user typed tells them apart.
2311+
matches = [record for record in candidates if record.get("server_url") == named]
2312+
if not matches:
2313+
matches = [record for record in candidates if _record_server_url(record) == target]
2314+
if not matches:
2315+
raise RSConnectException(
2316+
'No deployment record for server "%s" in %s.%s'
2317+
% (from_server, self.app_store.get_path(), _record_server_list(candidates))
2318+
)
2319+
if len(matches) > 1:
2320+
raise RSConnectException(
2321+
'Several deployment records match server "%s" in %s. Pass the record\'s URL '
2322+
"exactly.%s" % (from_server, self.app_store.get_path(), _record_server_list(matches))
2323+
)
2324+
return matches[0]
2325+
2326+
if not candidates:
2327+
return None
2328+
if len(candidates) == 1:
2329+
return candidates[0]
2330+
raise RSConnectException(
2331+
"Several deployment records exist in %s. Use --from-server to choose which one to "
2332+
"migrate.%s" % (self.app_store.get_path(), _record_server_list(candidates))
2333+
)
2334+
2335+
def migrate_to_connect_cloud(
2336+
self,
2337+
content_id: str,
2338+
from_server: Optional[str] = None,
2339+
overwrite: bool = False,
2340+
) -> AppMetadata:
2341+
"""Point this content's local deployment record at existing Posit Connect Cloud content.
2342+
2343+
Nothing is copied and no bundle is uploaded: the content must already exist in
2344+
Connect Cloud. What changes is the local record, which is the only way back to a
2345+
content item — Connect Cloud cannot look content up by name, so deploying
2346+
without a record creates a duplicate instead of updating the existing item.
2347+
2348+
The record migrated from supplies the title and app mode, and is removed
2349+
only when it is a shinyapps.io record, whose content has been migrated away
2350+
and whose record is therefore dead. A record for a Posit Connect server is
2351+
kept: its content still exists and is still deployable.
2352+
2353+
:param content_id: the id of the Connect Cloud content to point at.
2354+
:param from_server: the URL of the deployment record to migrate, needed only
2355+
when the local records cover several servers.
2356+
:param overwrite: replace an existing Connect Cloud record for this account.
2357+
:return: the record that was written.
2358+
"""
2359+
if not isinstance(self.remote_server, ConnectCloudServer) or not isinstance(self.client, ConnectCloudClient):
2360+
raise RSConnectException("Migrating a deployment record requires a Posit Connect Cloud account.")
2361+
2362+
source = self.migration_source_record(from_server)
2363+
2364+
# Checked before any request, so a run that cannot write anything makes no
2365+
# network calls and leaves the existing record untouched. The name-keyed
2366+
# location counts as existing too: a deploy reads it when the id-keyed one is
2367+
# missing, so a record there is this account's current target, and writing the
2368+
# id-keyed record would silently retarget the deploy and strand a duplicate.
2369+
target_key = self.record_server_key()
2370+
fallback_key = self.record_server_key_fallback()
2371+
existing = self.app_store.get(target_key)
2372+
if existing is None and fallback_key:
2373+
existing = self.app_store.get(fallback_key)
2374+
if existing and not overwrite:
2375+
raise RSConnectException(
2376+
'A Posit Connect Cloud deployment record for account "%s" already exists in %s, for content %s. '
2377+
"Use --overwrite to replace it."
2378+
% (self.remote_server.account_name, self.app_store.get_path(), existing.get("app_id"))
2379+
)
2380+
2381+
service = ConnectCloudService(self.client, self.remote_server)
2382+
with self.client:
2383+
content = self.client.get_content(content_id)
2384+
account_id = content.get("account_id")
2385+
account = service.account_for_id(account_id) if account_id else None
2386+
account_name = account["name"] if account else None
2387+
if not account or not account_name:
2388+
raise RSConnectException(
2389+
"Unable to determine which Posit Connect Cloud account owns content %s. "
2390+
"You may not have a role on that account." % content_id
2391+
)
2392+
# The record is keyed by account, and a deploy only reads the record for the
2393+
# account it is publishing to -- and would refuse content owned by another
2394+
# account anyway. A record written under the wrong account would therefore
2395+
# be silently ignored, so name the account that makes it usable instead.
2396+
# Compared by id when one is known, since that is what the key uses and what
2397+
# survives a rename; by name otherwise, which is then what the key uses too.
2398+
if self.remote_server.account_id:
2399+
same_account = account_id == self.remote_server.account_id
2400+
else:
2401+
same_account = account_name == self.remote_server.account_name
2402+
if not same_account:
2403+
raise RSConnectException(
2404+
'Content %s belongs to the Posit Connect Cloud account "%s", not "%s". '
2405+
"Re-run with -A %s." % (content_id, account_name, self.remote_server.account_name, account_name)
2406+
)
2407+
# An account id saved with a nickname skips get_account_by_name in
2408+
# validate_connect_cloud_server, so without this check a viewer role is
2409+
# caught only by the deploy -- after the source record has been removed.
2410+
if not service.can_publish_to(account):
2411+
raise RSConnectException(
2412+
'You have access to the Posit Connect Cloud account "%s" but do not have '
2413+
"permission to publish to it, so the next deploy from this directory would "
2414+
"fail. Ask an account administrator for the publisher role." % account_name
2415+
)
2416+
2417+
title = content.get("title") or (source or {}).get("title") or self.title
2418+
self.app_store.set(
2419+
target_key,
2420+
abspath(self.path),
2421+
self.remote_server.urls().content_url(account_name, content_id),
2422+
content_id,
2423+
None, # Connect Cloud content has no GUID separate from its id.
2424+
title,
2425+
# Connect Cloud derives the app mode from the content type and primary file,
2426+
# so there is nothing to read back from it; the source record's mode is kept
2427+
# when there is one. "unknown" does not block a later deploy of any mode.
2428+
(source or {}).get("app_mode") or AppModes.UNKNOWN.name(),
2429+
)
2430+
2431+
# Only a shinyapps.io record is removed, and only after the new record is
2432+
# safely written: leaving both is recoverable, losing both is not. Its
2433+
# content is what was migrated away, so the record is dead. A record for
2434+
# any other server still points at live content that is still deployable,
2435+
# and deploying one directory to both Connect and Connect Cloud is a
2436+
# supported setup, so that record is kept.
2437+
if source and _is_shinyapps_record(source):
2438+
self.app_store.remove(source["server_url"])
2439+
# The name-keyed record is the same account's, now superseded by the id-keyed
2440+
# one -- the same migration a deploy's write performs. The keys coincide when
2441+
# the account's name equals its id, and removing it then would delete the
2442+
# record just written.
2443+
if fallback_key and fallback_key != target_key:
2444+
self.app_store.remove(fallback_key)
2445+
2446+
record = self.app_store.get(target_key)
2447+
if record is None: # pragma: no cover - just written above
2448+
raise RSConnectException("The deployment record could not be saved.")
2449+
return record
2450+
22392451
@property
22402452
def supports_verify_before_activate(self) -> bool:
22412453
"""Whether the target server supports deploying a bundle as a draft and
@@ -3529,6 +3741,32 @@ def account_id(self) -> str:
35293741
account = self._client.get_account_by_name(self._server.account_name)
35303742
return account["id"]
35313743

3744+
def account_for_id(self, account_id: str) -> Optional[ConnectCloudAccount]:
3745+
"""The account with this id, among those the caller has a role on.
3746+
3747+
None means the account is not one of them, which for content the caller is
3748+
working with means they likely cannot publish to it either.
3749+
"""
3750+
for account in self._client.get_accounts():
3751+
if account.get("id") == account_id:
3752+
return account
3753+
return None
3754+
3755+
def account_name_for_id(self, account_id: str) -> Optional[str]:
3756+
"""The name of the account with this id, or None if the caller has no role on it."""
3757+
account = self.account_for_id(account_id)
3758+
return account["name"] if account else None
3759+
3760+
@staticmethod
3761+
def can_publish_to(account: ConnectCloudAccount) -> bool:
3762+
"""Whether the caller may publish to this account, by the same rule as
3763+
get_account_by_name, so a viewer role is judged identically either way.
3764+
3765+
Called on the class, not on self._client, so the rule holds for a stubbed
3766+
client too.
3767+
"""
3768+
return ConnectCloudClient._can_publish(account)
3769+
35323770
def content_url(self, content_id: str, account_id: Optional[str]) -> str:
35333771
"""Build the browsable URL for a content item.
35343772
@@ -3541,10 +3779,7 @@ def content_url(self, content_id: str, account_id: Optional[str]) -> str:
35413779
# be stale after a rename, and account ids are what survive one.
35423780
account_name = self._server.account_name
35433781
if account_id:
3544-
for account in self._client.get_accounts():
3545-
if account.get("id") == account_id:
3546-
account_name = account["name"]
3547-
break
3782+
account_name = self.account_name_for_id(account_id) or account_name
35483783
return self._server.urls().content_url(account_name, content_id)
35493784
except RSConnectException as exc:
35503785
# A URL we cannot build must not mask an otherwise successful deploy.

0 commit comments

Comments
 (0)