To be discussed: Reset staging portal, and use identical property Ids

[physikerwelt]

I find that using staging comes with some overhead, as it uses different properties. I suggest resetting the content on staging and copying the content from production to staging. This would also allow importing data from staging to production.

@MarcoReidelbach @AurelaShehu @bsch63 @bjschembera are you using the staging portal? What do you think about this idea? I would like to reach a consensus among all users before moving forward with the idea and discussing an implementation plan.

[MarcoReidelbach]

fine for me

[AurelaShehu]

I find this a good idea and is fine with me.

Maybe send an email around before you do so? (Just in case someone else is also using the staging portal). I think for example László (from TA3/WIAS) was using at some point the staging portal.

[bjschembera]

Also fine with me! However we're right now prototyping the import of some models with the staging portal, so it would be good to know some days in advance when this will happen

[physikerwelt]

Notes to maintainers:

Use the --wiki param when updating

root@staging-wikibase-56bb456488-9jc65:/var/www/html/w# ./maintenance/run update --wiki staging_wiki
MediaWiki 1.47.0-wmf.7 Updater

Your composer.lock file is up to date with current dependencies!
Going to run database updates for staging_wiki

Create admin account

root@staging-wikibase-56bb456488-9jc65:/var/www/html/w/maintenance# ./run createAndPromote.php --interface-
admin --sysop $MW_ADMIN_NAME $MW_ADMIN_PASS --wiki staging_wiki

Import data

kubectl cp MaRDI+portal_properties_templates_modules-20260626074014.xml staging-wikibase-56bb456488-9jc65:/var/www/html/MaRDI_import.xml -c mediawiki
./run importDump.php ../../MaRDI_import.xml --wiki staging_wiki
./run rebuildrecentchanges.php --wiki=staging_wiki 

[physikerwelt]

I have created a preview
https://p2.staging.mardi4nfdi.org
based on the current state of https://doi.org/10.5281/zenodo.20925418

[bsch63]

A reset is also fine with me. Sorry for the late reply but I was on vacations ...

[eloiferrer]

What is the current status here @physikerwelt ?

Is the plan still to reset staging.mardi4nfdi.org? If so, I think we should try to keep the user credentials https://staging.mardi4nfdi.org/wiki/Special:ListUsers

[physikerwelt]

@eloiferrer I was waiting for you as I got stuck with my p2 experiment. It would be nice if we could support different Wikibase repos and have different endpoints for each Wikibase instance. Do you think that's realistic? Currently, we have support for multiple wikibase client repos. For example opensearch uses the wiki parameter in the index name.

[eloiferrer]

I think it could work at the wikibase layer (maybe we should define another Wikibase.php just for staging, since this is getting confusing)

But it could maybe be problematic if we want to have the full functionality for each instance (with wdqs, qlever, and prefect/importer) since these would consume quite a bit of our resources. I am not sure how much configuration it would require for the kafka-based jobrunner.

PD. Shouldn't we also update my_wiki here

[physikerwelt]

I think it could work at the wikibase layer (maybe we should define another Wikibase.php just for staging, since this is getting confusing)

What exactly is confusing?

But it could maybe be problematic if we want to have the full functionality for each instance (with wdqs, qlever, and prefect/importer) since these would consume quite a bit of our resources. I am not sure how much configuration it would require for the kafka-based jobrunner.

Would the need for hardware not scale with the amount of data? I think that things like staging can always consume fewer resources (almost none) if there is only a little data. The Kafka-based jobrunner was the point when I got a bit confused.

PD. Shouldn't we also update my_wiki here
That would be a lot of effort; this is the key for a lot of things, and it had to be changed in probably 50 million places.

[eloiferrer]

What I find confusing is that there are many conditional statements to deal with databases that only apply to our staging configuration (there is my_wiki for production, my_wiki for staging, staging_wiki). And we have to deal with that in several places like

$mardiRepoDb = $wgDBname === 'staging_wiki' ? $wgDBname : 'my_wiki';

We are also hard-coding URLs:

if ( $wgDBname === 'staging_wiki' ) {
	$wikibaseHost = 'https://p2.staging.mardi4nfdi.org';
    $wgServer = 'https://p2.staging.mardi4nfdi.org';
    $wgCanonicalServer = 'https://p2.staging.mardi4nfdi.org';
}

which also only apply to staging.

And I also find confusing that we define the same repoDatabase for all instances but then we use different client databases.

$wgWBClientSettings['entitySources'] = [
		'mardi_source' => [
				'repoDatabase' => 'my_wiki',

That's is why I suggested having all this staging-specific configuration under LocalSettings.d/staging so that it does not affect the production instance if something goes wrong.

The hardware probably scales with the amount of data but there are also minimum requirements to get it just running. I think having a second staging instance would at least require having an additional:

• Qlever instance
• Qlever updater
• Qlever UI
• RDF streamer

which would require around 5Gi of RAM. We would also double the amount of topics in kafka/eventstreams/eventgate so maybe these would require more resources too.

I think if we want to keep staging and production as similar as possible it would be easier to have just one staging instance where we import the same properties as in production. But I missed why we want to have more than one staging instance.