We analyze the fluctuations in particle positions and inter-particle forces in disordered crystals in the limit of weak disorder. Such athermal jammed systems are fundamentally different from their thermal counterparts, characterized by constrained fluctuations of the bond angles and forces perpendicular to the lattice directions. We develop a disorder perturbation expansion in polydispersity about the crystalline state, which we use to derive exact results to linear order. We show that constrained fluctuations result as a consequence of local force balance conditions, and are characterized by non-Gaussian distributions which we derive exactly. The decoupling between the angular degrees of freedom and the magnitudes of the forces allows us to analytically predict several properties including the variation of the average coordination with polydispersity and packing fraction. We verify our results with numerical simulations using soft disks interacting via linear Harmonic potentials.