Is it possible to build a REST API in a Canister?

Yes. You can implement http_request in your canister and service requests that way.

Currently only query calls that don’t mutate state are natively supported (not update).

I made a pull request to change that but was asked to wait for a proper proposal process to be in place.

See for details.