Do we dream like Dall-E?

Dezember 12, 2022

Dreaming is still a little understood phenomenon of the human psyche, although there is a steady but slow uptake in releated research [1,2]. Especially its purpose remains somewhat a mystery and many suggetions have been put forth [3,4].

In this short article I do not want to sketch a general answer to all these questions nor speculate on the nature and function of dreams. I rather want to spend some time exploring the a certain aspect of dreams, that is their ‘detail-specificity’.

When dreams are being analysed in studies subjects are asked to describe the content of their dreams. This mostly works for REM sleep stages as other stages (for example SWS) are not memorized at all or only very occasionally [1:89].

But, interestingly enough their descriptions are on the one hand very vivid and describe specific events on the other hand most of the nitty-gritty details of the scence seem not to be ‘recallable’ (Like which specific brand and model a car was in a dream). Or the details are somewhat unrealistic or distorted (phenomens like ‘palinopsia’, ‘polyopia’, ‘macropsia’ or ‘micropsia’ etc.) [5:29].

This seems oddly familiar when looking at the results of (text-to-)image generating models like DALL-E (https://github.com/openai/DALL-E) or Stable Diffusion (https://github.com/CompVis/stable-diffusion). While detailed in on respect, they lake details in other areas of the picture or there are severe ‘mistakes’ or ‘distortions’.

1_fTforOofP-pmSrvWAJJlLg

a photo of a woman with a hat (generated using https://huggingface.co/spaces/stabilityai/stable-diffusion — first pass)

As seen in this examplary image generated using Stable Diffusion provided by the demo space of HuggingFace, the picture does depict a woman wearing a hat, but the properties and details are somewhat ‘off’ or deranged, esp. the hands are completely unproportional.

Or take the following image. Although there seem to be some features present which resemble parts of a piano it is by no means a correct representation of a piano on a mountain.

a piano on a mountain (generated using https://beta.dreamstudio.ai/dream first pass)

Similarly, in many of our dreams distorted or even impossible situations and objects occur, like a pink elephant or a flying human [6]. It could be the case that the neural nets of humans and that of the artificial kind could emply similiar imaginative mechanisms hinted at or exposed by the malfunctions of their image generation proccesses.

So, do humans really dream like graphic generating DNN models? Or the other way around, do DNNs dream like human?

That would carry the analogy too far and thus the question has to be answered with a clear ‘no’. For once lake of details in human dreams does not mean that there are frequent misconceptions of basic concepts. Whereas sometimes models generate humans with 4 or 6 fingers per hand, a human probably wouldn’t dream of such (except when dreaming specifically about the abnormalties of a human body).

Likewise the most recent wave of models like DALL-E 2 and Stable Diffusion are able to create such detailed scenary which a human could hardly dream (or at least more often than not the details of a dream remain highly selective and one would not be able to describe a detailed picture of a complex city scene (which is an easy feat for the aforementioned models).

But, analysing such behavior might further the understanding of the image generating capacity and mechanism of a human neural net. For instance, I do believe that the lack of details is based in large part on the fact that human dreams are more releated to imagination than to perception. In that regard their might be some similiarities between humans and DNNs as a DNN can be really realistic when it comes to sceneary it has seen very often, it can get rather scetchy on topics it isn’t familiar with — like the text of a computer screen in the background of an image.

a photo of an office with a turned on computer screen displaying the agile manifesto (generated using https://beta.dreamstudio.ai/dream selection from multiple passes)
___

[1] Nir, Y. and Tononi, G.: Dreaming and the brain: from phenomenology to neurophysiology. Trends in Cognitive Sciences. 2010 Feb;14(2):88–100.

[2] Wamsley, E.: Dreaming, waking conscious experience, and the resting brain: report of subjective experience as a tool in the cognitive neurosciences. Frontiers in Psychology. 2013;(4).

[3] Franklin, M. S. and Zyphur, M. J.: The Role of Dreams in the Evolution of the Human Mind. Evolutionary Psychology. 2005;3(1).

[4] Wamsley, E.: Dreaming as Constructive Episodic Future Simulation. Sleep. 2021;(44,Supplement 2):A15.

[5] Schwartz, S. and Maquet, P.: Sleep imaging and the neuro-psychological assessment of dreams. Trends in Cognitive Sciences. 2002; 6(1): 23–30.

[6] Hobson, J. A., et al.: The neuropsychology of REM sleep dreaming. Neuroreport (1998);9:R1–14.