The sound from environment will be altered after it transmits through headphones to the ear canal. A hear-though filter can be designed and implemented in the headphones to create a more natural hearing experience, i.e., offer a transparent mode for headphones. The design of hear-through filter is also required in some other applications, e.g., augmented reality audio. In this paper, a constrained hear-through filter design approach is proposed. It is firstly shown that the hear-through filter design problem can be formulated in a similar form to active noise control filter design in the frequency domain. One advantage of this design approach is that multiple practical constraints can be applied conveniently by formulating a constrained optimization problem. Then the constrained optimization problem for hear-through filter design is reformulated as cone programming problem which can be solved efficiently. The proposed design approach can also specify the desired delay of reproduced sound. The designed filter can be directly implemented in an active noise control system in the headphone such that the requirement for extra electronic components can be minimal.