MIDDLEBURY — The Middlebury selectboard on Tuesday unanimously endorsed the concept of removing the current municipal building at the intersection of College and South Main streets and replacing it with a more efficient structure that will include town offices and a community center.