Repository navigation
Are Many-to-Many link supported with fastapi? #121
Copy link
Copy link
Closed
Labels
questionFurther information is requestedFurther information is requested
Description
First Check
- I added a very descriptive title to this issue.
- I used the GitHub search to find a similar issue and didn't find it.
- I searched the SQLModel documentation, with the integrated search.
- I already searched in Google "How to X in SQLModel" and didn't find any information.
- I already read and followed all the tutorial in the docs and didn't find an answer.
- I already checked if it is not related to SQLModel but to Pydantic.
- I already checked if it is not related to SQLModel but to SQLAlchemy.
Commit to Help
- I commit to help with one of those options 👆
Example Code
from typing import List, Optional
from fastapi import Depends, FastAPI, HTTPException, Query
from sqlmodel import Field, Relationship, Session, SQLModel, create_engine
sqlite_file_name = "test.db"
sqlite_url = f"sqlite:///{sqlite_file_name}"
# version sans fastapi: engine = create_engine(sqlite_url, echo=True)
connect_args = {"check_same_thread": False}
engine = create_engine(sqlite_url, echo=True, connect_args=connect_args)
def create_db_and_tables():
SQLModel.metadata.create_all(engine)
class MPartMagnetLink(SQLModel, table=True):
"""
MPart/Magnet many to many link table
"""
magnet_id: Optional[int] = Field(
default=None, foreign_key="magnet.id", primary_key=True
)
mpart_id: Optional[int] = Field(
default=None, foreign_key="mpart.id", primary_key=True
)
class MagnetBase(SQLModel):
"""
Magnet
"""
name: str
class Magnet(MagnetBase, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
mparts: List["MPart"] = Relationship(back_populates="magnets", link_model=MPartMagnetLink)
class MagnetRead(MagnetBase):
id: int
class MagnetCreate(MagnetBase):
pass
class MagnetUpdate(SQLModel):
"""
Magnet
"""
name: str
mparts: List["MPart"] = [] #Relationship(back_populates="magnets", link_model=MPartMagnetLink)
class MPartBase(SQLModel):
"""
Magnet Part
"""
name: str
class MPart(MPartBase, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
magnets: List[Magnet] = Relationship(back_populates="mparts", link_model=MPartMagnetLink)
class MPartRead(MPartBase):
id: int
class MPartCreate(MPartBase):
pass
class MPartUpdate(SQLModel):
"""
Magnet Part
"""
name: str
magnets: List[Magnet] = []
class MPartReadWithMagnet(MPartRead):
magnets: List[MagnetRead] = []
class MagnetReadWithMParts(MagnetRead):
mparts: List[MPartRead] = []
def get_session():
with Session(engine) as session:
yield session
app = FastAPI()
@app.patch("/magnets/{magnet_id}", response_model=MagnetRead)
def update_magnet(
*, session: Session = Depends(get_session), magnet_id: int, magnet: MagnetUpdate):
db_magnet = session.get(Magnet, magnet_id)
if not db_magnet:
raise HTTPException(status_code=404, detail="Magnet not found")
magnet_data = magnet.dict(exclude_unset=True)
for key, value in magnet_data.items():
setattr(db_magnet, key, value)
session.add(db_magnet)
session.commit()
session.refresh(db_magnet)
return db_magnet
@app.on_event("startup")
def on_startup():
create_db_and_tables()Description
- Start uvicorn
- Open browser: it crashes:
2021-10-04 15:53:27,962 INFO sqlalchemy.engine.Engine COMMIT
INFO: Application startup complete.
INFO: 127.0.0.1:54084 - "GET /docs HTTP/1.1" 200 OK
INFO: 127.0.0.1:54084 - "GET /openapi.json HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/vscode/.local/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 375, in run_asgi
result = await app(self.scope, self.receive, self.send)
File "/home/vscode/.local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
return await self.app(scope, receive, send)
File "/home/vscode/.local/lib/python3.8/site-packages/fastapi/applications.py", line 208, in __call__
await super().__call__(scope, receive, send)
File "/home/vscode/.local/lib/python3.8/site-packages/starlette/applications.py", line 112, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/vscode/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
raise exc from None
File "/home/vscode/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
await self.app(scope, receive, _send)
File "/home/vscode/.local/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
raise exc from None
File "/home/vscode/.local/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
await self.app(scope, receive, sender)
File "/home/vscode/.local/lib/python3.8/site-packages/starlette/routing.py", line 580, in __call__
await route.handle(scope, receive, send)
File "/home/vscode/.local/lib/python3.8/site-packages/starlette/routing.py", line 241, in handle
await self.app(scope, receive, send)
File "/home/vscode/.local/lib/python3.8/site-packages/starlette/routing.py", line 52, in app
response = await func(request)
File "/home/vscode/.local/lib/python3.8/site-packages/fastapi/applications.py", line 161, in openapi
return JSONResponse(self.openapi())
File "/home/vscode/.local/lib/python3.8/site-packages/fastapi/applications.py", line 136, in openapi
self.openapi_schema = get_openapi(
File "/home/vscode/.local/lib/python3.8/site-packages/fastapi/openapi/utils.py", line 387, in get_openapi
definitions = get_model_definitions(
File "/home/vscode/.local/lib/python3.8/site-packages/fastapi/utils.py", line 24, in get_model_definitions
m_schema, m_definitions, m_nested_models = model_process_schema(
File "pydantic/schema.py", line 548, in pydantic.schema.model_process_schema
File "pydantic/schema.py", line 589, in pydantic.schema.model_type_schema
File "pydantic/schema.py", line 241, in pydantic.schema.field_schema
File "pydantic/schema.py", line 440, in pydantic.schema.field_type_schema
File "pydantic/schema.py", line 773, in pydantic.schema.field_singleton_schema
File "pydantic/schema.py", line 667, in pydantic.schema.field_singleton_sub_fields_schema
File "pydantic/schema.py", line 495, in pydantic.schema.field_type_schema
File "pydantic/schema.py", line 839, in pydantic.schema.field_singleton_schema
File "/usr/lib/python3.8/abc.py", line 102, in __subclasscheck__
return _abc_subclasscheck(cls, subclass)
TypeError: issubclass() arg 1 must be a class
Operating System
Linux
Operating System Details
Ubuntu 20.04 LTS
SQLModel Version
0.0.4
Python Version
Python 3.8.10
Additional Context
I've tried to adapt the tutorial example with the many-to-many associative tables
for using it with fastapi without success. If this feature is supported it would be great
to have an example in the tutorial.
Best
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
questionFurther information is requestedFurther information is requested
Activity
[-]Does Many-to-Many link are supported with fastapi[/-][+]Are Many-to-Many link supported with fastapi?[/+]res234 commentedon Oct 5, 2021
I've noticed a similar issue with delayed annotation from Pydantic. Copying the documentation exactly, the following code works. Here is my
models.pyfile. This code runs well:If you split the models into
teams.pyandheroes.pythen the same that Trophime is getting occurs.Here's
teams.py:And here's
heroes.py:The issue can even be replicated in a single
models.pyfile:All of these produce the same exception. If you start uvicorn and then open your browser and go to the docs page you get the following exception:
I have tried this on Python 3.8.1 and Python 3.9.7. I think that Pydantic had an issue earlier that was fixed in a previous version: pydantic/pydantic#1298.
mafetoro92 commentedon Oct 5, 2021
I am experiencing the same exception as well.
LucidDan commentedon Oct 6, 2021
Prefacing this by saying I'm relatively new to pydantic and sqlalchemy, so take what I suggest with a grain of salt.
For @Trophime 's sample code - If you add this at the bottom of the file, it works:
This should be done after all the router endpoints are defined.
It resolves this particular case, by updating the forward reference to the actual referenced model after everything has loaded.
For multiple files it seems to be a bit trickier -
You can do some non-PEP8 shenanigans like importing the forward ref models at the bottom of the file, and then update the forward ref - technically this can work, although your python linter will hate you:
If you don't want to engage in acts of dubious code style legality, you can also use a package file to import all the models together, and run any updates there...this is how I'm doing it for my current project. Something like:
Note though, the above code caused me some NameErrors, unless I explicitly passed the namespace for the forward refs, like this:
I haven't done a lot of testing on this; that issue could be specific to my code. I mostly figured this out by reading pydantic code, github issues, and applying brute force trial and error.
I'm not at all sure this is a good solution, but it is a workaround that seems to have sorted the issue for me, at least...for now. 😅
res234 commentedon Oct 6, 2021
@LucidDan thank you so much! I did tinker around using
update_forward_refs()but I ran into NameErrors as well. Your solution of explicitly passing the the namespace gets around this!I will keep on trying this with more complex models and see if I run into any issues. Thank you so much.
Trophime commentedon Oct 7, 2021
@LucidDan thanks for your help.
Now I have a stupid question as a beginner in this field.
How can I actually proceed to update
mpartsof amagnetin my example?I've naively tried something like that:
which is obviously not working
I think the simplest way is to use a method to add mpart from an mpart_id.
Otherwise I guess I need to modify the updat_mpart method...
Thanks for your suggestions
lobotmcj commentedon Oct 18, 2021
The package file import all (e.g.,
__init__.py) described above by @LucidDan works for my more complex case, but only with theForwardRefModel=ForwardRefModeland you have to do this for every forward reference...including ones in parent classes (even if you've already updated their forward refs).For example, if the models in one of your files are like this:
...then you have to (or I had to at least) update forward refs as follows to avoid NameErrors and get the docs to actually load:
note that I already updated
EventReadWithHosts->HostRead, but had to do it again for theEventReadFullsubclassI'm sure I am misunderstanding/describing something incorrectly, but it didn't work for me until I did the above
res234 commentedon Oct 19, 2021
@lobotmcj Yes, I found the exact same thing. You have to update the forward refs for every forward reference. I may be doing something wrong as well though!
LucidDan commentedon Oct 19, 2021
Yeah to be clear folks, this is what I had to do, too. I only used a single example, but in my actual project, It wasn't just one model, it was several, and several namespace entries that had to be passed in as well.
How I approached finding the ones to add was I set my server up so it would immediately raise a runtime error and crashing out if there was any models that didn't resolve properly (rather than only logging a caught error when you accessed an endpoint), and then just added each of those models causing issues to the list of updates.
I could've perhaps run an iteration over all models and done an update on all of them but that seemed a bit excessive...it's a large project, I've got many dozens of models.
It's not ideal, for sure...definitely want to see a better solution, and I'll eventually put some time into figuring it out, at least for my own project. I'll update this issue if I make any progress on that, would love to hear from anyone else that figures out better solutions, too.
masreplay commentedon Jan 6, 2022
any updates?
res234 commentedon Jan 7, 2022
@masreplay What updates were you looking for specifically? I think @LucidDan offered a solution.
thomas-daughters commentedon Feb 25, 2022
@LucidDan's approach worked for me too, but isn't ideal. As a minimum, the tutorial / documentation should be updated. Better still would be if this could somehow be managed seamlessly by the library itself.
invokermain commentedon Apr 11, 2022
Yeah this is pretty complicated stuff, none of the above solutions seem 'good'. Relying on 'on import' behaviour always feels funny. But you can't solve it in the ASGI lifespan startup either as its too late by that point. I wonder if SQLModel can somehow update forward refs when they are added to the metadata.
Theoretically, it should be possible to iterate over the fields and go 'I can update this I know what it is'. But also quite hacky.
It sucks that the recommendation is to make 4+ models for each database table (if doing a simple CRUD app etc). So a medium size app with maybe 20 tables all of a sudden has a file with 80+ classes in it because you can't really solve the circular import issue 😆
At a minimum the documentation should be updated to address that this issue exists, the use TYPE_CHECKING solution doesn't actually fix the issue.
plocher commentedon Apr 27, 2022
This seems to be much worse than simply a "all models need to be in the same file" or "fix with update_forward_refs"...
It also means you can't really abstract the CRUD or ROUTE code into per-table modules if there is any cross referencing, as then the importing becomes runtime circular.
Using the above modular example with the code from the tutorial, try to refactor the code paths as well:
The code in hero,.py needs to refer to Team, and the code in team.py needs to refer to Hero...
How does one refactor these examples so that it is possible to create a Team and add a Hero to it (or create a Hero on a Team...) with per-table MODEL, CRUD and ROUTE files?
Alternatively, is there a good "large project" design pattern for keeping the business logic / database logic abstraction modular in the face of these relationships?
(I really like the ActiveRecord concept from Boris Lau in issue 254 that adds class methods to the SQLModel classes for the CRUD functionality, but this limitation seems to make it impractical for these cross reference cases)
Sancho66 commentedon Aug 2, 2022
Any updates on these issue ? it's really frustrating to not find a solution...
Zaffer commentedon Aug 25, 2022
I am experiencing the same complexity, would love a more elegant solution.
Eryx5502 commentedon Feb 16, 2023
Is there any on-going work on this? As @invokermain suggested, it should be possible to modify the
SQLModelMetaclassso that every time a model is defined itsForwardRefs are updated if possible and every other model depending on the newly defined class is also checked. Not sure if this would be the neatest approach though, but other similar approaches based onSQLModelMetaclassshould be doable.For now, as a workaround I'm doing what @LucidDan suggests and using the
__init__.pyfile of my models module for updating the forward refs providing the namespace. But since doing it manually it's such a pain, I am generating a dict with all the models (everything that inherits from SQLModel) and doing.update_forward_refson every of them using that dict to provide a namespace with every needed model. Of course this is not optimal since I'm not iterating only over those models withForwardRefs, but its easier this way. It should be possible to inspect the models and generate a list of the ones with forward refs to iterate on those only.Here is the code I'm using in case it helps anyone:
udiNur commentedon Mar 4, 2023
@Eryx5502 Thanks for providing your solution. It works well for me while using models in different files and allows me to maintain a direct import from the parent model. Here's the updated code block:
models/parent.py
models/children.py
One of them needs to actually import the other model (in that case the parent imports the children), and then the second can do something like
if TYPE_CHECKING:, otherwise (when non of them actually using the importdkubatko commentedon Aug 12, 2023
Thank you! This actually works. This should definitely be addressed in the module itself. Just curious, is there an actual downside to doing it this way? My understanding is that it doesn't matter if you update references for packages that don't need it.
DamianLiu237 commentedon Jul 7, 2024
Not sure when it was added but with the addition of
.model_rebuild(), this solution resolved the annotation issue for me and seems the most "elegant".redb0 commentedon Aug 29, 2024
I also faced this problem. I'm using Python 3.9. Calling
model_rebuildin__init__didn't help me. TheNameError: name error still occurs '...' is not defined, if you use an annotationList['Model']Only the option mentioned here #757 (comment), but he looks terrible.AimoreRRD commentedon Sep 27, 2024
Remember to initialize (import) the models like it is said in the tutorial and docs (that is what worked very well for me):
https://sqlmodel.tiangolo.com/tutorial/code-structure/?h=circular#app-file