Rottnest Island was inhabited by Aborigines approximately 30,000 years ago, until rising sea levels separated the island from the mainland of Western Australia approximately 7,000 years ago. The island features in Nyoongar Aboriginal mythology and was called Wadjemup. There are Aboriginal artifacts on the island that have been dated from 6,500 to more than 30,000 years ago.

The island was observed by various Dutch sailors from 1610, including Frederick de Houtman in 1619, and the three ships Waekende Boey, the Elburg and the Emeloort in 1658. The name was given to the island the island by the Captain Willem de Vlamingh on 29 December 1696. He mistaked the Rottnest Island Quokka's large rats and called it"rattenest" meaning "rat's nest" in the Dutch language.

Other explorers that stopped at the island included the French expedition of Nicholas Baudin in 1801 and 1803, Phillip Parker King in 1822, and Captain James Stirling in 1827.

Upon the establishment of the British colony in nearby Perth, ten Aboriginal prisoners were sent to Rottnest Island in August 1838. The Colonial Secretary announced in June 1839 that the island would become a penal establishment for Aboriginal people, and between 1838 and 1931, Rottnest Island was used as an Aboriginal prison to "pacify" "local natives". In "pacifying" an Aboriginal population, men were rounded up and chained for offences ranging from spearing livestock, burning the bush or digging vegetables on what had been their own land. It has been estimated that there may be as many as 369 Aboriginal graves on the island. Except for a short period between 1849 and 1855 during which the prison was closed, some 3,700 Aboriginal men and boys, from all parts of the state, were imprisoned.

The island became more or less devoted to recreational use in the 1900s, aside from a brief period of exclusive military use during World War II. Wartime cannon batteries and camouflaged sites are still in existence at various parts of the island.

One of the most little known aspects of Rottnest's history, even to Perth locals, is that was the home of an internment camp in both World War I and World War II. In WWI it was mostly used for German and Austrian enemy aliens, but was closed towards the end of the war due to appalling living conditions. During WWII it was used exclusively for Italian enemy aliens. This too was closed about halfway through the war, and its occupants were sent to various internment and work camps, with some finding themselves as far away as New South Wales. Many of the internees held at these camps had been law-abiding citizens of Western Australia for many years.

Another aspect of World War II was the placement of large guns in the middle of Rottnest Island for defence of the Fremantle port, and the construction of a railway between the jetty at Thompson Barracks at the eastern end of the island and the guns for the transport of materials for the guns.

After WWII the guns and the buildings related to them were either removed or abandoned, and the railway was removed. In the 1990s the guns and the railway were extensively reconstructed for use as a tourist attraction. However parts of disused military installations and older structures on the island have issues due to their construction containing asbestos.