Can I disable Algolia on some store ?

Yes you just need to disable indexing for the store where you do not need Algolia. See the screenshot bellow