The congregation of the Port Carling Anglican Church began around 1870 when a clergyman from Bracebridge and local lay readers led services in settler's homes. The congregation started building a church in 1877 and it was completed in 1885. Pine benches were donated to the church by the Governor General Lord Dufferin and the rectory added in 1897; however, on June 13, 1912, the hotel next door, known as the Stratton House, caught fire and spread to the church. The local Orange Lodge let the congregation use their meeting hall as a temporary church until the new church was finished and consecrated on August 26, 1913. The east stained glass window, which was dedicated to the first clergyman, and the pine benches donated by the Governor General were saved from the original building and installed in the new construction. The remainder of the stained glass windows were created by McCausland's of Toronto. A bell was added in 1914.

St. James the Apostle Anglican Church is a rectangular wood frame construction with vinyl siding, a stone foundation, stone and vinyl-covered buttresses, a steep pitched gable roof, a small addition to the northeast corner, and a square bell tower on the southwest corner. The west elevation of the main structure has a central arched window with a wood window casing while the west elevation of the bell tower has a central entrance with a contemporary double door and arched transom, and arched wood shutters fit into a window opening near the top of the tower. The north and south elevations of the main structure have three arched windows, and the south elevation of the bell tower has an arched window and arched wood shutters fit into a window opening near the top of the tower. The front of the property, near the sidewalk, has a river-stone fence.