Cloud cover is the main physical factor limiting the downward shortwave (SW) solar radiation flux. In modern models of climate and weather forecasts, physical models describing radiative transfer through clouds may be used. However this option computationally expensive. Instead, one may use parameterizations which are simplified schemes for approximating environmental variables. The purpose of our study is to assess the capabilities of machine learning models of approximating radiation flux based on all-sky optical imagery in order to assess the links between observed cloud cover properties with the flux. We applied various machine learning (ML) models: classic ML models and convolutional neural networks (CNN). These models were trained using the dataset of all-sky optical imagery accompanied by SW radiation flux measurements. The Dataset of All-Sky Imagery over the Ocean (DASIO) is collected in Indian, Atlantic and Arctic oceans during several expeditions from 2014 till 2021. When training our CNN, we applied heavy source data augmentation in order to force the CNN to become invariant to brightness variations and, thus, approximating the relationship between the visual structure of clouds and SW flux. We demonstrate that the CNN supersedes existing parameterizations known from literature in terms of RMSE. Our results allow us to assume that one may acquire downward shortwave radiation flux directly from all-sky imagery. We also demonstrate that CCNs are capable of estimating downward SW radiation flux based on clouds' visible structure.