API Star as a framework-independant tool


#1

I’ve been moving ahead with work on removing the server from API Star, originally pitched here.

The pull request is here.

This would mean that the next version of API Star wouldn’t include the server at all. Instead it’d be targeted at being framework-independent API tooling, including API documentation, schema validation, API mocking, a dynamic API client library, and a type system that can map to or from JSON schema, and render to HTML forms.

This isn’t going to be massively popular with folks using the framework, but I’ve not been maintaining it, and I’m far happy with a greatly scoped-down project, that can be used with any Python framework, and that fits in much better with other work that I look after. (eg. I want to be able to start recommending apistar for generating API docs for REST framework, and using it as the api client library, while gradually deprecating coreapi)

Alongside the dependency injection, one of my main aims in releasing API Star was to help push ASGI forward. I’ve since put in a bunch of work to a far more tightly scoped project, Starlette that I think provides a far more well-designed approach for an async web framework/toolkit than API Star has been able to (since API Star has the weird mix of being able to support either WSGI or ASGI).

There’s a few different options for continuity of the server bit of the project…

  1. Do nothing. The existing apistar versions won’t disappear so you just stay pinned to them. (Possibly we continue to accept a critical bug PRs against a 0.5.x branch and roll releases against that.)
  2. Have a fork of the project, under a different name. The only concrete changes here are that someone else would be doing the maintainance other than me, and that projects would need to change the import name to update.
  3. Forget about the DI layer, but make sure that there is suitable documentation and support for using apistar 's type system and API docs together with starlette.
  4. I start a new project that combines apistar 's types and API docs, with starlette for the base framework, and just adds in the DI layer and API docs. (ie. a continuation of apistar's server, but with a clear set of boundaries between the different projects.) I’m not especially keen on this right now, since I’ve got a lot of other plates spinning, but it’s not something I’ve ruled out.

Happy to take feedback on the above, but one way or another I’m going to continue pushing forward with this, and need to make sure everyone’s aware of where my own maintenance priorities lie.

Thanks,

Tom


#2

TL/DR

I suggest Option 0. But make it clear on the README.md and docs that it is frozen and direct people to starlette for the latest and greatest.

What brought me to apistar was the the auto documentation and dependency injection. I wasn’t really looking for an async framework, but it was great to see the project supported both programming paradigms so that I could grow into the other when I understood the nuisances of programming async in python. I personally didn’t need async in my work projects because I was very comfortable with WSGI and liked how it dropped the baggage of other frameworks and was python 3.5+ . I wanted to develop just APIs and not MVC web pages.

I had some disagreements with my partner though around the breaking changes, your proposed new direction of the project, and the maintenance of the project. I eventually had to agree to freeze our code that was using apistar and move back to something more stable (he chose Flask which I had experience in already).

I think if your priorities are starlette then maybe you let apistar die and you take the parts that worked and incorporate them into your new work. For the people that want the experience I described above our friend @Bogdanp took many of your concepts from apistar to produce the Molten framework which could reasonably be considered the WSGI cousin of apistar.

For the people who want the ASGI experience then it sounds like starlette is going to be the new thing. Even heard Michael Kennedy talking on Python Bytes about building an ASGI web course around starlette which I am looking forward to purchasing.

I’m sticking around for Starlette so that I can learn the ASGI ways. I have learned a lot from this and you guys continue to teach me more with each interaction.


#3

After our discussion about that I’ve been thinking in creating a package for Starlette with the dependency injection layer, the type system and the endpoints for generating the schema and docs. Obviously I’m not going to do it from scratch, my idea was to extract the injector from APIStar and shape it into a middleware-like. For the schema and docs views, I’ll adapt it somehow into Starlette way.

I don’t know if that even could works, but I’ll try it when I have some free time.


#4

Option 2! Or even half of it! What help do you need?

I would love to be able to use Starlette with the type system. Specifically, being able to declare route function parameters with Python types, and because of that having validation, sensible errors and OpenAPI automatic generation (just the JSON endpoint), is just genius. I don’t even really care about the generated docs as I can spin another Swagger UI container (or integrate Swagger UI directly) and make it consume the generated OpenAPI JSON endpoint.

I loved that idea and I’ve been very interested in migrating my Flask / Flask-apispec / Marshmallow / Webargs stacks to API Star and helping to build stuff around it (plug-ins, authentication, Docker stuff).

I understand the way you are directing the project now. But I would love to know how could that be achieved, and what help do you need, if there’s any I can provide.


#5

Related discussion: https://github.com/encode/apistar/issues/627

Option 2!

Actually yes, I’m pretty keen on that. Or rather not just focusing on using API Star’s type system with Starlette, but focusing on things we can do to bring it to a bunch of different frameworks as the validation and schema generation component for them.

I’d suggest that we’re likely to want to encourage patterns that are less opinionated that API Star’s previous “use the types in the function signature”, since that just doesn’t play well with existing frameworks, and ends up dictating the framework design far too heavily.

An example of the kind of integration that we could work towards in order to play well with frameworks such as Flask would be to do something like use decorators to enforce validation and annotate views in such a way as to allow automated schema generation.

@query_params(
    search=String(max_length=100),
    score=Integer(min_value=0, max_value=10)
)
@response_body(MyType)

It might also be worth taking a look at existing schema generation tools such as https://github.com/rochacbruno/flasgger to see how they currently support marshmallow integration, and do something that aligns well with that.

Ultimately, finding good ways to do this sort of thing without having to force an entire framework on users is part of why I’m taking this more tightly scoped approach.


#6

Hey all !

As I mentioned in my previous response, I’m currently working on adapt the server part of APIStar to work with Starlette. My initial idea was to reproduce exactly the same functionality that we currently have in APIStar v0.5 as a layer on top of Starlette, so it’s on early stages but seems that is possible to do.

To me, there are two functionalities I want to keep in Starlette:

  1. Components.
  2. Types system with Schema generation.

Following that discussion I’ve been thinking in generate a couple of pieces to replace APIStar server functionality and to be usable with Starlette:

  1. What I have in mind is to define a proper dependency injection system to handle Components in Starlette, so we can reuse all of those pieces that we created for our services (such as connectors to databases, redis…). In my opinion, APIStar mechanism for handlind those Components provides awesome conditions to create a large plugins ecosystem, so I want to keep that for Starlette as well. I have thatt almost done, so we can see it soon.

  2. Regarding to Types and Schemas, I can tackle it as well, but would be nice to define the way we want to follow. My first idea was to use the previous point and add all APIStar types and validators to work the same way it’s working in v0.5, but I’m open to do it in a more agnostic way like the example that @tom proposed.


#7

I see, cool.

It seems like flasgger generates OpenAPI schemas based on docstrings and OpenAPI YAML files, etc. right?

Have you seen Flask-apispec? It integrates Marshmallow and Webargs (and is built by the same guys). The syntax is actually very, very similar to what you are describing. What I like about it (compared to flasgger) is that it’s a schema generated based on code, instead of code / validation / serialization based on a pre-defined schema annotated on docstrings or somewhere else. Similar to what API Star was doing.

That’s actually what I’ve been using up to now. Here’s a snippet of one example route: https://github.com/tiangolo/full-stack/blob/master/{{cookiecutter.project_slug}}/backend/app/app/api/api_v1/endpoints/user.py#L62

@docs.register
@doc(description="Create new user", security=security_params, tags=["users"])
@app.route(f"{config.API_V1_STR}/users/", methods=["POST"])
@use_kwargs(
    {
        "email": fields.Str(required=True),
        "password": fields.Str(required=True),
        "first_name": fields.Str(),
        "last_name": fields.Str(),
        "group_id": fields.Int(required=True),
    }
)
@marshal_with(UserSchema())
@jwt_required
def route_users_post(
    email=None, password=None, first_name=None, last_name=None, group_id=None
):
    current_user = get_current_user()

    if not current_user:
        ...

But actually, I really liked the idea of not having to declare a parameter in multiple places (in the decorator and as a function parameter). And having everything done by the Python typing system. And getting better IDE support / tools because they know the types. I think that idea (that I’ve only seen in API Star) was brilliant. It solves several problems at the same time, improves code manageability / enjoyability, reduces the amount of lines to achieve something, doesn’t require remembering the order of several decorators to work properly, etc. Declaring a couple types, with Python standard syntax, and then getting validation, API documentation, error reporting, serialization and much better IDE integratiion and error analysis. With just a couple types in the function.

But yes, it’s true that it would be a more tightly coupled “framework” than a set of independent tools.

On the other side, Flask already has Flask-apispec, with Marshmallow, Webargs, which provide a very similar set of features to what you are describing. Not based on Python types. But if the idea is to do not use types, then it would be pretty much the same, no? Or am I missing something?

Also, I’ve been thinking recently that Flask, with all the context managers, global proxy objects, etc. is great for learning the basics of web development, but extending it, using all these global proxies, context managers, etc. becomes quite more daunting (at least to me). And IDE integration is very difficult, e.g. it’s almost impossible to know the attributes and methods of the request object without going directly to the documentation… and then, on the other side, there was API Star with very explicit types everywhere, I really liked it.

…just to put my opinion there. And to give you some acknowledgment of how brilliant your idea (API Star’s idea) was. :slight_smile:

Now, knowing that Starlette has some of the best performances out there (competing even against Go), but that you don’t want to make it a tightly coupled framework, the best way to get a functionality similar to API Star would be to create a little project (“framework”) that inherits from Starlette’s classes, right? I guess similar to what @PeRDy is doing? Or do you have any suggestion or direction to point me to? I don’t have any knowledge of ASGI yet…

Also, thinking about independent pieces, have you guys checked Pydantic? It’s based on Python types too (contrary to Marshmallow) and seems to have quite good performance. I know it was mentioned in a thread but I don’t know if it had anything that would make it unusable by something like API Star or a Starlette based extension. Do you have any opinion on that?


#8

@PeRDy do you have a public repo with your work? I’d like to star it to remind me to check it out once I get some time. :smiley:


#9

Yep, it’s a complete mess right now, but I just pushed the Dependency Injection working for Components over http views, endpoints and websockets. I’m just playing around, so it’s far to be ready for production usage.

Here is the repository: https://github.com/PeRDy/starlette-api

I’m working now in adding the type system through that dependency injection, so that the functionality should be similar to what we currently have in APIStar. That could be a temporal solution while we decide the path to follow for defining type system and schema generation.


#10

@PeRDy thanks for sharing it!


#11

@PeRDy, I’m not that much of a python developer, but if you need help to implement or discuss ideas, ping me!
I love API Star a little bit too much, and I’d like to adapt django_apistar to whatever the successor of the server implementation may be.
now, I’m focusing on studying molten to see if I can make it happen, but it would be nice to have an asgi counterpart as well :slight_smile: