We don't really know. A recent manuscript submitted to Nature shows minute persistence in various tissues as long as 230 days or more post-infection. [1] RNA is fragile, if the infection is completely cleared or not integrated into the genome in some way we would not expect it to persist.

[1] https://assets.researchsquare.com/files/rs-1139035/v1_covere... (Figure 1)