We proposed an image fusion method based on cognition for infrared and visible images. Multi-source image fusion is guided top-down by using the prior knowledge as acquired from deep understanding of multi-source images. As different fusion methods and fusion rules are used for contents of different importance, it makes the image fusion process more free and flexible. Experiments for the presented method are conducted by using infrared and visible image. The fusion results and the image fusion quality indexes are compared with those of the traditional region-based fusion methods. The comparison show the proposed method is far superior to the traditional region-based fusion methods.