Shaykh Ibn Taymiyyah (d. 728 H) states:

We do not agree that ‘Ali was the overall best of his time. Rather, the best of this Ummah after its Prophet are Abu Bakr, then ‘Umar, as is authentically narrated from ‘Ali and others.1
It is true that our Sunni brothers consider Abu Bakr to be the best of our Ummah, followed only by ‘Umar. However, during the lifetime of the Messenger of Allah, sallallahu ‘alaihi wa alihi, it was a different story entirely. There is irrefutable evidence in the Sunni books establishing that the Sahabah used to consider Amir al-Muminin, ‘alaihi al-salam, to be their best during the lifetime of the Messenger. Imam Ahmad (d. 241 H) presents one of such proofs:

‘Abd Allah (b. Ahmad b. Hanbal) – my father (Ahmad b. Hanbal) – Muhammad b. Ja’far – Shu’bah – Abu Ishaq – ‘Abd al-Rahman b. Yazid – ‘Alqamah – ‘Abd Allah (b. Mas’ud):
“We used to say that the overall best of the people of Madinah was ‘Ali b. Abi Talib.”2
“We” (in the hadith)3 apparently refers to the Sahabah generally, and more specifically to the most senior of them living in Madinah. Ibn Mas’ud was obviously making a reference to a past which was then different from the present. This was why he said “we USED TO”. In other words, at that point in time when he was making his statement, things had become different. People were now giving ‘Ali’s place to another person. Ibn Mas’ud was, no doubt, speaking about the time of the Prophet. All the most senior Sahabah and their neighbours were living in Madinah with the Messenger of Allah. The phrase “people of Madinah” originally referred to them (excluding only the Prophet, of course).4 These, needless to say, included Abu Bakr, ‘Umar and ‘Uthman.
So, is the above report authentic? Al-Hafiz (d. 852 H) says about the first narrator:

‘Abd Allah b. Ahmad b. Muhammad b. Hanbal al-Shaybani, Abu ‘Abd al-Rahman: son of the Imam, thiqah (trustworthy).5
What about his father? Al-Hafiz answers:

Ahmad b. Muhammad b. Hanbal b. Hilal b. Asad al-Shaybani al-Maruzi, a Baghdad resident, Abu ‘Abd Allah: One of the Imams, thiqah (trustworthy), hafiz, jurist, hujjah (an authority).6
Al-Hafiz also has these comments about the third narrator:

Muhammad b. Ja’far al-Hazali al-Basri, better known as Ghandar: Thiqah (trustworthy), sahih al-kitab (i.e. ahadith from his books are sahih) except that there was some negligence in him.7
Whatever negligence he had does not affect his ahadith from Shu’bah at all. He used to accurately record the latter’s reports. So, he narrated them from his books with perfect precision. Al-Hafiz provides further information in this respect:

Ibn Mahdi said: “We used to benefit from the books of Ghandar on Shu’bah. Waki’ named him sahih al-kitab.” Abu Hatim narrated from Muhammad b. Aban al-Balakhi that Ibn Mahdi said: “Ghandar is more accurate than me as far as Shu’bah is concerned.” Ibn al-Mubarak said, “When the people disagree about the hadith of Shu’bah, the book of Ghandar used to judge between them.” Ibn Abi Hatim said: “I asked my father about Ghandar and he replied, ‘He was saduq (very truthful), and was a teacher and in the hadith of Shu’bah, he is thiqah (trustworthy).’”8
The fourth narrator, Shu’bah, is a pillar of Sunni ahadith. Al-Hafiz gives the catch-phrases about him:

Shu’bah b. al-Hajjaj b. al-Ward al-‘Atki, their freed slave, Abu Busṭam al-Wasiṭi, al-Basri: Thiqah (trustworthy), hafiz, extremely precise. Al-Thawri used to say: “He was the amir al-muminin (the supreme leader) in al-Hadith.”9
Abu Ishaq al-Sabi’i is the fifth narrator, and al-Hafiz has this to say about him:

‘Amr b. ‘Abd Allah b. ‘Ubayd .... Abu Ishaq al-Sabi’i: Thiqah (trustworthy); narrated a lot (of ahadith), a great worshipper (of Allah), from the third (ṭabaqat). He became confused (in his narrations) during the end part of his lifetime.10
Of course, Shu’bah heard from him before the memory loss. ‘Allamah al-Albani (d. 1420 H) explains:

Sufyan al-Thawri and Shu’bah also narrated from Abu Ishaq, although both did not mention the Descent. The reports of both of them (from Abu Ishaq) are more authentic, because they both heard from him BEFORE he became confused.11
Another relevant point is that Abu Ishaq is a mudalis and has, on the surface, narrated the report of Ibn Mas’ud above is an ‘an-‘an form from ‘Abd al-Rahman b. Yazid. However, the tadlis does NOT, in reality, affect the ‘an-‘an reports of Abu Ishaq – among others - as long as it is Shu’bah narrating from him. Allamah al-Albani states further:

Al-Tirmidhi said: “A hasan sahih hadith, al-Thawri and Shu’bah narrated it from (‘an) Abu Ishaq.”
I say: It is (truly hasan sahih) as he (al-Tirmidhi) has stated, and both of them (i.e. al-Thawri and Shu’bah) narrated from him (i.e. Abu Ishaq) before his confusion. As for Shu’bah, he never narrated anything from him (i.e. Abu Ishaq) except what he (Abu Ishaq) explicitly stated to have directly heard from the person he is narrating from (i.e. tahdith), as stated in his tarjamah (biography). Due to his (Shu’bah’s) narration from him (i.e. Abu Ishaq), the problem of his tadlis is removed.12
In a clearer word, whenever Shu’bah narrates from Abu Ishaq (as in this case of Ibn Mas’ud’s hadith), all the problems associated with the latter’s reports are removed. The former narrated from him before his confusion in his ahadith, and never transmitted any tadlis-infested reports from him. So, whenever Shu’bah narrates an ‘an-‘an report from Abu Ishaq, there actually is tahdith by the latter from his Shaykh. The ‘an-‘an form is only Shu’bah’s convenience style. No wonder, Imam al-Bukhari (d. 256 H) includes this sanad in his Sahih:

Sulayman b. Harb – Shu’bah – Abu Ishaq – ‘Abd al-Rahman b. Yazid.13
This is an ‘an-‘an report by Abu Ishaq from ‘Abd al-Rahman b. Yazid (the same Shaykh as in the athar of Ibn Mas’ud). Nevertheless, Imam al-Bukhari considers the chain to be sahih.
Imam Ahmad b. Hanbal has also documented a similar ‘an-‘an chain:

‘Abd Allah (b. Ahmad b. Hanbal) – my father (Ahmad b. Hanbal) – Yazid – Shu’bah – Abu Ishaq – Abu Maysarah.14
Al-Arnauṭ comments:

Its chain is sahih upon the standard of the two Shaykhs.15
‘Allamah al-Albani too authenticates yet another ‘an-‘an chain of Abu Ishaq:

Its chain: Hafs b. ‘Umar – Shu’bah – Abu Ishaq – al-Aswad – ‘Abd Allah.
I say: This chain is sahih upon the standard of al-Bukhari.16
Imam Abu Ya’la (d. 307 H) also documents an ‘an-‘an chain by Abu Ishaq, from ‘Abd al-Rahman b. Yazid, like al-Bukhari:

Ishaq – ‘Abd al-Samad – Shu’bah – Abu Ishaq – ‘Abd al-Rahman b. Yazid – al-Aswad17
Shaykh Dr. Asad gives this verdict:

Its chain is sahih18
Let us now move to the sixth narrator in the sanad of Ibn Mas’ud’s athar: ‘Abd al-Rahman b. Yazid. The status of ‘Abd al-Rahman as a thiqah (trustworthy) narrator of Sahih al-Bukhari is already well-known. Nonetheless, we are pleased to present this further confirmation by al-Hafiz:

‘Abd al-Rahman b. Yazid b. Qays al-Nakha’i, Abu Bakr al-Kufi: Thiqah (trustworthy).19
Finally, concerning the seventh and last narrator (‘Alqamah), al-Hafiz al-‘Asqalani proclaims with full strength:

‘Alqamah b. Qays b. ‘Abd Allah al-Nakha’i al-Kufi: Thiqah (trustworthy), thabt (accurate), faqih (a jurist), ‘abidun (a great worshipper of Allah).20
With this, it is absolutely clear and undeniable that Ibn Mas’ud’s report that the Sahabah used to consider ‘Ali as the overall best among them has an impeccably sahih chain. All the narrators are thiqah, and the chain is fully and perfectly connected.
Even then, the same athar has been recorded with a second sahih chain in that same Fadhail al-Sahabah:

‘Abd Allah (b. Muhammad b. ‘Abd al-‘Aziz al-Baghwi) – my grandfather (Ahmad b. Muni’ al-Baghwi) – Abu Qaṭan – Shu’bah – Abu Ishaq – ‘Abd Allah b. Yazid – ‘Alqamah – ‘Abd Allah b. Mas’ud:
“We used to say that the overall best of the people of Madinah was ‘Ali b. Abi Talib.”21
We already know the status of Shu’bah, Abu Ishaq and Alqamah. So, let’s find out about these new names.
This is al-Hafiz’s verdict on the first narrator of this new sanad:

‘Abd Allah b. Muhammad b. ‘Abd al-‘Aziz, Abu al-Qasim al-Baghwi: Al-hafiz, al-saduq (the extremely truthful), the top scholar of his time.... I (al-‘Asqalani) say: He has been declared thiqah (trustworthy) by al-Daraqutni, al-Khatib and others. Al-Khatib said, “He was thiqah (trustworthy), accurate, and narrated a lot (of ahadith)”.... I (al-‘Asqalani) say: The man is absolutely thiqah (trustworthy).22
Concerning his grandfather, al-Hafiz further submits:

Ahmad b. Muni’ b. ‘Abd al-Rahman, Abu Ja’far al-Baghwi....: Thiqah (trustworthy), hafiz.23
Abu Qaṭan too is thiqah (trustworthy), as confirmed by al-Hafiz:

‘Amr b. al-Haytham b. Qaṭan ... Abu Qaṭan al-Basri: Thiqah (trustworthy).24
Of course, ‘Abd Allah b. Yazid was a junior Sahabi, and therefore needed no investigation. He is automatically thiqah (trustworthy). Al-Hafiz states:

‘Abd Allah b. Yazid b. Zayd b. Husayn al-Ansari al-Khaṭmi: a junior Sahabi. He was the wali (ruler) of Kufah for Ibn al-Zubayr.25
So, we have a second impeccable sanad for the hadith.

References
1. Abu al-‘Abbas Ahmad b. ‘Abd al-Halim b. Taymiyyah al-Harrani, Minhaj al-Sunnah al-Nabawiyyah (Muasassat Qurtubah; 1st edition, 1406 H) [annotator: Dr. Muhammad Rashad Salim], vol. 6, p. 475
2. Abu ‘Abd Allah Ahmad b. Hanbal al-Shaybani, Fadhail al-Ṣahabah (Beirut: Muasassat al-Risalah; 1403 H) [annotator: Dr. Wasiyullah Muhammad ‘Abbas], vol. 2, p. 604, # 1033
3. We are calling it a hadith, rather than an athar, because it lays down a consensus of the Ṣahabah, which they had during the lifetime of the Prophet of Allah. It is our firm belief that they could not have formed such a consensus except on the basis of what they had learnt from the Messenger.
4. See Qur’an 9:101 and 9:120
5. Ahmad b. ‘Ali b. Hajar al-‘Asqalani, Taqrib al-Tahdhib (Beirut: Dar al-Maktabah al-‘Ilmiyyah; 2nd edition, 1415 H) [annotator: Mustafa ‘Abd al-Qadir ‘Ata], vol. 1, p. 477, # 3216
6. Ibid, vol. 1, p. 44, # 96
7. Ibid, vol. 2, p. 63, # 5805
8. Shihab al-Din Ahmad b. ‘Ali b. Hajar al-‘Asqalani, Tahdhib al-Tahdhib (Dar al-Fikr; 1st edition, 1404 H), vol. 9, p. 85, # 129
9. Ahmad b. ‘Ali b. Hajar al-‘Asqalani, Taqrib al-Tahdhib (Beirut: Dar al-Maktabah al-‘Ilmiyyah; 2nd edition, 1415 H) [annotator: Mustafa ‘Abd al-Qadir ‘Ata], vol. 1, p. 418, # 2798
10. Ibid, vol. 1, p. 739, # 5081
11. Abu ‘Abd al-Rahman Muhammad Nasir al-Din b. al-Hajj Nuh b. Tajati b. Ādam al-Ashqudri al-Albani, Ṣahih Abi Dawud (Kuwait: Muasassat al-Gharas li al-Nashr wa al-Tawzi’; 1st edition, 1423 H), vol. 7, p. 410, # 2387
12. Abu ‘Abd al-Rahman Muhammad Nasir al-Din b. al-Hajj Nuh b. Tajati b. Ādam al-Ashqudri al-Albani, Silsilah al-Ahadith al-Ṣahihah wa Shayhun min Fiqhihah wa Fawaidihah (Riyadh: Maktabah al-Ma’arif li al-Nashr wa al-Tawzi’; 1st edition, 1415 H), vol. 5, p. 481, # 2366
13. Abu ‘Abd Allah Muhammad b. Isma’il b. Ibrahim b. Mughirah al-Bukhari al-J’ufi, al-Jami’ al-Ṣahih al-Mukhtasar (Beirut: Dar Ibn Kathir; 3rd edition, 1407 H) [annotator: Dr. Mustafa Dib al-Bagha], vol. 3, p. 1373, # 3551
14. Abu ‘Abd Allah Ahmad b. Hanbal al-Shaybani, Musnad (Cairo: Muasassat Qurtubah) [annotator: Shu’ayb al-Arnaut], vol. 6, p. 182, # 25532
15. Ibid
16. Abu ‘Abd al-Rahman Muhammad Nasir al-Din b. al-Hajj Nuh b. Tajati b. Ādam al-Ashqudri al-Albani, Ṣahih Abi Dawud (Kuwait: Muasassat al-Gharas li al-Nashr wa al-Tawzi’; 1st edition, 1423 H), vol. 5, p. 150, # 1267
17. Abu Ya’la Ahmad b. ‘Ali b. Muthanna al-Mawsili al-Tamimi, Musnad (Damascus: Dar al-Mamun li al-Turath; 1st edition, 1404 H) [annotator: Dr. Husayn Salim Asad], vol. 8, p. 35, # 4541
18. Ibid
19. Ahmad b. ‘Ali b. Hajar al-‘Asqalani, Taqrib al-Tahdhib (Beirut: Dar al-Maktabah al-‘Ilmiyyah; 2nd edition, 1415 H) [annotator: Mustafa ‘Abd al-Qadir ‘Ata], vol. 1, p. 596, # 4057
20. Ibid, vol. 1, p. 687, # 4697
21. Abu ‘Abd Allah Ahmad b. Hanbal al-Shaybani, Fadhail al-Ṣahabah (Beirut: Muasassat al-Risalah; 1403 H) [annotator: Dr. Wasiyullah Muhammad ‘Abbas], vol. 2, p. 646, # 1097
22. Shihab al-Din Abu al-Fadhl Ahmad b. ‘Ali b. Hajar al-‘Asqalani, Lisan al-Mizan (Beirut: Manshurat Muasassat al-A’lami li al-Matbu’at; 2nd edition, 1390 H), vol. 3, pp. 338-339, # 1339
23. Ahmad b. ‘Ali b. Hajar al-‘Asqalani, Taqrib al-Tahdhib (Beirut: Dar al-Maktabah al-‘Ilmiyyah; 2nd edition, 1415 H) [annotator: Mustafa ‘Abd al-Qadir ‘Ata], vol. 1, p. 47, # 114
24. Ibid, vol. 1, p. 748, # 5146
25. Ibid, vol. 1, p. 547, # 3715