Woods and Brazee established their mill on Conway Bayou in 1857 with a 3,000-ft daily capacity. The genesis of this mill may have been operating as early as 1853, for W. T. Block wrote in Cotton Bales, Keelboats and Sternwheelers: A History of the Sabine River and Trinity River Cotton Trades, 1837-1900 that by 1853 sawmill operations near Orange included those of John Merriman, R. A. Neyland, R. H. Jackson, Brosser Wood and Co., and the Empire Mills.
The mill was not enumerated in the Manuscript Census Returns of 1860. Brazee was killed at Pine Bluff, Arkansas, in 1863, during the Civil War. Alexander Gilmer bought a controlling interest in the mill about 1867; the mill could produce about 7,000 feet daily. It burned down in 1867, and Woods and Gilmer rebuilt it, using the latest in circular saw technology, pushing cutting capacity to 15,000 feet daily. By the later part of the 1870ss, the mill could cut 20,000 feet of lumber daily and 25,000 crossties.
The firm of Moore &Swinford bought the mill on Conway Bayou in 1877, and the large Orange Lumber Company plant would be built on this site. With Swinford's purchase of the D. R Wingate interest of the Phoenix mill of the Wingate-Swinford operation 1877 or 1879, the Phoenix was known as the “Upper Mill” while the Conway Bayou mill was the “Lower Mill.”