How can I change the string column type to Unicode?

[phi-friday]

First Check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the SQLModel documentation, with the integrated search.
• I already searched in Google "How to X in SQLModel" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to SQLModel but to Pydantic.
• I already checked if it is not related to SQLModel but to SQLAlchemy.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from typing import Any, Callable, Optional, Union

import orjson
from sqlalchemy.types import Unicode
from sqlmodel import Field, SQLModel

__all__ = (
    "create",
    "read",
    "update",
    "data",
)


def orjson_dumps(value: Any, *, default: Callable[[Any], Union[Any, None]]) -> str:
    return orjson.dumps(value, default=default).decode()


class basemodel(SQLModel):
    class Config(SQLModel.Config):
        json_loads = orjson.loads
        json_dumps = orjson_dumps


class base(basemodel):
    name: str = Field(
        ..., index=False, nullable=False, sa_column_kwargs={"type_": Unicode}
    )
    json_data: str = Field(
        ..., index=False, nullable=False, sa_column_kwargs={"type_": Unicode}
    )


class create(base):
    pass


class update(base):
    pass


class read(base):
    pass


class data(base, table=True):
    id: Optional[int] = Field(default=None, primary_key=True)

Description

In mssql, the names and json_data columns are made of varchar.
Since I want an nvarchar column, I specified type_ separately in the sa_column_kwargs option.
However, I confirmed that an error occurred in sqlalchemy.
This error seems to occur when the positional argumet and keyword argumet for type_ overlap.
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/sql/schema.py", line 1586, in __init__
raise exc.ArgumentError(
sqlalchemy.exc.ArgumentError: May not pass type_ positionally and as a keyword.
In this case, I would appreciate it if you could tell me what to do.

Operating System

Linux

Operating System Details

in docker,
image: python:3.9-buster

SQLModel Version

0.0.4

Python Version

3.9.8

Additional Context

No response

[RobertRosca]

Looked into this a bit.

When you pass sa_column_kwargs those get propagated through to when the sa column is created here:

https://github.com/tiangolo/sqlmodel/blob/02da85c9ec39b0ebb7ff91de3045e0aea28dc998/sqlmodel/main.py#L447

The issue happens since you're passing type_ = Unicode as a keyword argument, but SQLModel already passes sa_type through as the positional argument.

The sa_type variable is found by:

https://github.com/tiangolo/sqlmodel/blob/02da85c9ec39b0ebb7ff91de3045e0aea28dc998/sqlmodel/main.py#L373-L409

Which currently doesn't support Unicode. One option would be to add Unicode to the options supported by get_sqlalchemy_type, or to deal with the special case of receiving a type_ kwarg.

IMO allowing advanced users to pass through whatever they want via sa_column_kwargs is a better/more generic approach. I'll do a PR for that 👍

---

Edit: whoops this just occurred to me when I used it. Easier way to do this is to set the column type directly via sa_column:

from sqlalchemy import Column

...

class base(basemodel):
    name: str = Field(
        ..., index=False, nullable=False, sa_column=Column(Unicode)
    )
    json_data: str = Field(
        ..., index=False, nullable=False, sa_column=Column(Unicode)
    )

[phi-friday]

Thank you so much for letting me know a solution that I can use right away. Options such as max_length or nullable should be specified again in the sa_column, but it's not a big problem right now.
Thank you @RobertRosca! Thank you SQLModel!

[RobertRosca]

😁 glad to help!

So setting sa_column worked like you wanted? If so could you close the issue? I'll also look into doing a PR to add this use case to the docs, with Unicode as an example.

[phi-friday]

@RobertRosca
Of course!
Thank you!

[tiangolo]

In SQLModel 0.0.11 you can now set sa_type with the SQLAlchemy type. 🎉

[fbuccioni]

I have done this by monkey patching:

import sqlmodel.sql.sqltypes
from sqlalchemy import types

sqlmodel.sql.sqltypes.AutoString.impl = types.Unicode

Maybe it could be a global config system or flag to use Unicode in AutoStrings