The son of Thomas Gilbert of Ratcliffe, Buckinghamshire, a captain in the Royal Marines, by Elizabeth, daughter of William Long Nathaniel Hutton, rector of Maids Moreton, Buckinghamshire, was born near Burnham Beeches, Buckinghamshire, 14 May 1786, and educated at Manchester Grammar School from 1800. He was nominated to a school exhibition, and matriculated at Brasenose College, Oxford, on 30 May 1805. At the Michaelmas examination of 1808 he was placed in the first class in literis humanioribus, one of his four companions being Robert Peel. He graduated B.A. 16 January 1809, and succeeded to one of Hulme's exhibitions on 8 March following. Having been elected to a fellowship, he proceeded M. A. 1811, and B.D. 1819.
He was actively engaged for many years as a college tutor, and in 1816–18 was a public examiner. On the death of Frodsham Hodson in 1822, he was elected Principal of Brasenose on 2 February, and took his D.D. degree on 30 May. From 1836 to 1840 he was Vice-Chancellor of Oxford University. On the death of Philip Shuttleworth he was nominated to the bishopric of Chichester, 24 January 1842, and consecrated at Lambeth Palace on 27 February.
Gilbert took much interest in Lancing College and other educational institutions. Of high church opinions himself, he was averse to Catholic ceremonials. He took proceedings in the 1850s against John Mason Neale, Warden of Sackville College; and on 14 October 1868 he interdicted John Purchas from ritualism in services at St James's Chapel, Brighton. This latter case led to much litigation, and eighteen works were printed in connection with the matter.
Gilbert was married on 31 December 1822 to Mary Anne, only surviving child of the Rev. Robert Wintle, vicar of Culham, Oxfordshire. They had eleven children. Mary Anne died in the Bishop's Palace at Chichester on 10 December 1863.
Gilbert died of paralysis at the Bishop's Palace in Chichester on 21 February 1870, and was buried in Westhampnett Church, Sussex, on 25 February.