At present, the wildfire smoke detection algorithm based on YOLOv3 has problems, such as low accuracy and slow detection speed. In this article, we propose a cross-layer extraction structure and multiscale downsampling network with bidirectional transpose FPN (BCMNet) for fast detection of wildfire smoke. First, a cross-layer extraction module, which combines linear feature multiplexing and receptive field amplification, is designed. It can improve the speed and accuracy of wildfire smoke detection. Second, a multiscale downsampling module with different convolution kernels and maximum pooling operation is designed to preserve the details of the image while downsampling. Then, a bidirectional transposed FPN based on transposed convolution upsampling is designed. It can bidirectionally fuse visual features of shallow layer and semantic features of deep layer on the corresponding scale. The feature information flow between smoke feature maps of different resolution is emphasized. Finally, a wildfire smoke detection system of the Internet of Things based on BCMNet is built by combining the hardware and detection model. The experimental results show that the proposed method achieves 85.50% mAP 50 and 79.98% mAP 75 at 40 FPS on NVIDIA Geforce RTX 2080 Ti, which is superior to the common smoke detection methods.