R. G. Brown, the son of R. G. Brown, told The Gulf Coast Lumberman, in 1950, that the Brown Hardwood Company began in 1889 when his father and his father's friend, J. F. Judd, surveyed land on the west side of Longview, in Gregg County, for the possibility of going into the timber business. R. G. Brown then established a sawmill at Longview. Brown later established a sawmill at Carthage in 1917. Brown Hardwood Company had a circular sawmill in Carthage in 1928. The company also had a mill at Silsbee, Hardin County, that year. By 1931, the main mill was at Carthage, and retail outlets were at Kilgore, Longview, and Joinerville. When gas was discovered in the Carthage area in the 1940s, the wealth that it brought to Carthage and in Panola County led to a large expansion of the company's retailing business.
In 1952, reported The Gulf Coast Lumberman, the company was put up for sale. The sawmill plant included a cutting capacity of 25,000 feet daily, a dry kiln, a Woods 8-knife planer, moulder, two boilers, and three engines. It had been shut down for several weeks. The sawmill business was discontinued toward the end of the decade. In February, 1960, Brown Hardwood Company changed its name to Brown Building Services, Inc., to reflect more correctly its business. R. G. Brown II was president of the corporation.