The first Labor Day holiday was celebrated on Tuesday, September 5, 1882, in New York City, in accordance with the plans of the Central Labor Union. The Central Labor Union held its second Labor Day holiday just a year later, on September 5, 1883.
In 1884 the first Monday in September was selected as the holiday, as originally proposed, and the Central Labor Union urged similar organizations in other cities to follow the example of New York and celebrate a "workingmen's holiday" on that date. The idea spread with the growth of labor organizations, and in 1885 Labor Day was celebrated in many industrial centers of the country. (read full text at U.S. Department of Labor's website)