Queen Elizabeth House was founded as result of a gift of £100,000 given by Sir Ernest Oppenheimer to the University of Oxford. This money was donated for the development of colonial studies and the establishment of a colonial centre in Oxford. This donation was supplemented by a gift of £50,000 from the British Government's Colonial Development and Welfare Fund. QEH was constituted by Royal Charter in 1954 to provide a residential centre which people concerned with the study of Commonwealth affairs could visit to make contacts and exchange ideas.

On its foundation Queen Elizabeth House was a corporate establishment administered by a governing body comprising a President (Viscount Chandos), the Vice-Chancellor (A.L.P. Norrington) and up to 16 other members nominated by the Government or the University. It also had a warden who was essentially the manager of Queen Elizabeth House. Although affiliated to the University, QEH was not funded by it and relied on the Government, overseas governments, industry research grants and donations from private individuals for its funding.

In 1958 Queen Elizabeth House was extended to provide a hall and additional residential accommodation and in 1961 it was able to house the Institute of Commonwealth Studies, with which Queen Elizabeth House was closely linked. Funds for the extension were raised by appeals to industry. During the 1960s Queen Elizabeth House was developed as a residential and conference centre, a centre for University lectures and classes, and for social activities for advanced students and other visitors from the Commonwealth. Those residing at Queen Elizabeth House included government service cadets, journalists, teachers and students from Commonwealth countries. Queen Elizabeth House offered vacation courses as well as hosting the University's Overseas Service Course.

In the 1980s there was a drive to reshape Queen Elizabeth House as a centre for international studies, a move away from being purely a centre for study concerning the Commonwealth. In 1986 it was merged with the Institute of Commonwealth Studies and the Oxford University Institute of Agricultural Economics to create the International Development Centre, a department of the University within the Social Studies Faculty (Statutes 1987). As a result of this merger the structure, funding and governance of Queen Elizabeth House changed. Most of the functions of the governing body were assumed by the University. In 1994 the governing body was officially dissolved following the transfer of the assets of Queen Elizabeth House to the University and the surrender of the Royal Charter. It was replaced by an Advisory Council for the department.

From 1958 to 2005 Queen Elizabeth House was located at 20-21 St Giles. From the late 1950s to 1977 Queen Elizabeth House leased 7 and 9 Banbury Road which were used as residential units and living quarters for the Warden. In 2006, the department moved to the former School of Geography building in Mansfield Road and became known as the Oxford Department of International Development (Queen Elizabeth House).

Wardens/Directors ((From 1971 the position of Warden of Queen Elizabeth House) and Director of the Institute of Commonwealth Studies were held by the same person):

Presidents: