A couple of things come to mind: 1. embedding methods: there're a couple of ways to do that, the most used one is OpenAi's text-davinci-002, although in my use case(short sentence description of API) it didn't work pretty well; 2. how you truncate documents into pieces: for this langchain has some implementation and helpful pointers.

I think you have to do lots of experiment on this till you find your best information retrieval strategy