W. T. Block wrote in Cotton Bales, Keelboats and Sternwheelers: A History of the Sabine River and Trinity River Cotton Trades, 1837-1900 that by 1853 sawmill operations near Orange included those of John Merriman, R. A. Neyland, R. H. Jackson, Brosser Wood and Co., and the Empire Mills.
The Empire Mills sawmill was located, in 1856, about five miles south of Orange (then Madison) adjacent to the west bank of the Sabine River. The owner was C. A. Buckley of Galveston and the manager William Hewson. The mill had a ten-man crew. It is unknown if any of the workers were slaves.
The mill's destruction came as the result of violence. On May 31, 1856, Sam Ashworth and Jack Bunch killed Deputy Sheriff Samuel Deputy. The killers were pursued by a posse of Moderator Orange Vigilantes. When Ashworth and Bunch hid in a pile of stacked lumber at the mill site, the vigilantes deliberately set afire the stacked lumber. The mill burned down as a result. Losses were estimated to be $15,000, including 100,000 board feet of stacked lumber along with the mill.
Sam Ashworth apparently escaped while Jack Bunch was taken into custody, according to the Liberty Gazette article of November 17, 1856. A jury could not be obtained, so venue was changed to Jefferson County. The article reported that Ashworth had been arrested in Hill County and would be returned for trial.