There are growing needs to understand how extreme weather events impact the electrical grid. Renewable energy sources such as solar photovoltaics are expanding in use to help sustainably meet electricity demands. Wildfires and, notably, the widespread smoke resulting from them, are one such extreme event that can impair the performance of solar photovoltaics. However, isolating the impact that smoke has on photovoltaic energy production, separate from ambient conditions, can be difficult. In this work, we seek to understand and quantify the impacts of wildfire smoke on solar photovoltaic production within the Western United States. Our analysis focuses on the construction of a random forest regression model to predict overall solar photovoltaic production. The model is used to separate and quantify the impacts of wildfire smoke in particular. To do so, we fuse historical weather, solar photovoltaic energy production, and PM2.5 particulate matter (primary smoke pollutant) data to train and test our model. The additional weather data allows us to capture interactions between wildfire smoke and other ambient conditions, as well as to create a more powerful predictive model capable of better quantifying the impacts of wildfire smoke on its own. We find that solar PV energy production decreases 8.3% on average during high smoke days at PV sites as compared to similar conditions without smoke present. This work allows us to improve our understanding of the potential impact on photovoltaic-based energy production estimates due to wildfire events and can help inform grid and operational planning as solar photovoltaic penetration levels continue to grow.