Abstract The present paper concerns determination of boundary conditions for mathematical modelling of heat transfer. In particular, it covers determination of heat transfer coefficient ? during ambient air cooling of steel rails on cooling bed and during accelerated cooling of a part of rail web by submerging in still water. First sub-section covers measurement of cooling curves for both heat treatment processes. Eleven thermocouples placed in drilled holes have been used for the measurements. The following chapter contains calculation of the heat transfer coefficient based on heat flux obtained by analysis of the time dependence of gradient between two thermocouples placed closed to the rail surface. The mathematical-physical background of the calculation is given in a special sub-section of the study. The obtained results were compared with inverse analysis results. The inverse analysis involved inverse calculation by finite element method (with the aid of FormFEM software for forming process simulation) and processing of measured cooling curves to obtain a new relationship between heat transfer coefficient ? and temperature of the rail surface during cooling. The paper discusses the causes of the discrepancies between results (particularly during accelerated cooling in water) of both methods and between the theoretical values. There is a summary of the impact of formation of rail head steam cushion upon the amount of extracted heat. Inversion analysis results corrected with respect to generated phase transformation heat were used for mathematical modelling of the laboratory experiment in the initial setting. Resulting cooling curves were in excellent agreement with measured values.