An open source computational fluid dynamics (CFD) solver has been incorporated into the WindNinja modeling framework. WindNinja is widely used by wildland fire managers, as well as researchers and practitioners in other fields, such as wind energy, wind erosion, and search and rescue. Here, we describe the CFD solver and evaluate its performance against the WindNinja conservation of mass (COM) solver, and previously published large-eddy simulations (LES), for three field campaigns with varying terrain complexity: Askervein Hill, Bolund Hill, and Big Southern Butte. We also compare the effects of two model settings in the CFD solver, namely the discretization scheme used for the advection term of the momentum equation and the turbulence model, and provide guidance on model sensitivity to these settings. Additionally, we investigate the computational mesh and difficulties regarding terrain representation. Two important findings from this work are: (1) CFD solver predictions are significantly better than COM solver predictions at windward and lee side observation locations, but no difference was found in predicted speed-up at ridgetop locations between the two solvers, and (2) the choice of discretization scheme for advection has a significantly larger effect on the simulated winds than the choice of turbulence model.