A daily tally of new cases revealed two athletes tested positive in the Village and one elsewhere. They come a day after an unidentified person, who was not a competitor, became the first case in the Village.
The Olympic Village, a complex of apartments and dining areas, will house 6,700 athletes and officials at its peak when the 2020 Games, delayed last year over the pandemic, finally get underway.
The Tokyo Olympics, which will be held largely behind closed doors to prevent infections, are unpopular in Japan where opinion polls have consistently demonstrated a lack of support.
On Saturday, International Olympic Committee president Thomas Bach appealed for Japanese fans to get behind the Games, saying he was "very well aware of the scepticism" surrounding the event.