Benutzerin:Verena

...

KI-SEMINAR

Projektdokumentation

Kurzbeschreibung (EN)

THE OFFICE (working title), 2019
Humans and plants live on different timescales. This is certainly one of the reasons why in everyday life, plants might often seem static and object-like to us. »The Office« (2019) makes use of existing video footage that covers large time spans, in this case, popular long-term tv shows. These productions showcase office and apartment interiors over a longer period of time, often also including houseplants which are mainly used for decorative purposes. Some of these series run over years or even decades and thus comprise many hours of footage. Convolutional neural networks are used as a tool to detect scenes involving houseplants while processing large quantities of the given video material. Selected scenes are compiled into a time-lapse movie which documents plant growth over a long period of time. While the lively movement and growth of the normally passively seeming plants becomes visible, the human activities become blurry and fade into the background.
Keywords: plants, plant-human-relationships, time, timescales, time-lapse, video, neural networks

Hintergrund/Research

Projekte Pflanzen & KI

• Botanica Variegata (2019?) von Sjoerd Ter Borg

Projekte Environment & KI

• Biophillic Vision - Experiment 1: experimenting with machine learning to remove cars from video footage von Chris Harris

Technische Umsetzung/Vorgehensweise

Erste Schritte

• Convolutional Neural Networks for object detection (in videos)
• Heartbeat Tutorial Part 1: Detecting objects in videos and camera feeds using Keras, OpenCV, and ImageAI
  • using a Python library called ImageAi
  • using a pretrained YOLOv3 computer vision model that can recognize 80 different objects, including "potted plants"
  • TO DO: installing a number of python libraries and ImageAi
• Heartbeat Tutorial Part 2: Analyze and Visualize Detected Video Objects Using Keras and ImageAI

Aktueller Stand

Working with the pretrained network

• analyze video frame by frame (works for example with .mp4 and .m4v)
• detect custom objects, in this case of the category "potted plant"
• output a list called "detected frames" that for each frame in the video contains either a 0 or 1 to indicate whether a plant has or has not been detected in that frame. For example (for a video with 24 frames):

[0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0]

• transform this list into a numpy array
• transform this array into another array called "changeArray" only indicates the positions where something changes in the array (if values jump from 0 to 1, or from 1 to 0 or in other words detect the beginning and end of the plant sequence
• reshape array into an array called "changeArrayReshaped" with 2 columns with each row containing only the start and stop frame of the plant sequences

[[ 724 736]

[1716 1717]
[1734 1739]
[1742 1807]
[1809 1812]
[2073 2075]
[2077 2102]
[3260 3309]
[3344 3376]
[3416 3424]
[3497 3526]]

• write this data into a csv file

Nächste Schritte

• Trainingsmaterial generieren: aus den einzelnen extrahierten Sequenzen (je 1 Pflanze) Einzelbilder erzeugen

• train custom model based on collected images
• collecting and processing larger quantities of suitable video material
• further automatizing the process of preselecting plant scenes, splitting the

detected plants into different classes, and making a new cut based on these classes in order to generate visually coherent video material

• final selection of interesting output movies for presentation purposes
• video post production (if neccessary)

Links zum Thema KI

• Don’t let industry write the rules for AI
• The AI Text Generator That's Too Dangerous to Make Public

Evolutionary Algorithms

• Evolutionary algorithm outperforms deep-learning machines at video games, MIT Technology Review, 06/2018

BLOCKCHAIN READING GROUP notes

> seminar page: Blockchain Reading Group

Bitcoin & Blockchain Basics

• What is Bitcoin? A Beginner’s Guide, coinsutra, 10/2018
• Blockchain Explorer (live)

Ethereum

• We Met The Founder Of Ethereum, VICE on HBO, 04/2018
• "Code For Ethereum’s Proof-of-Stake Blockchain to Be Finalized Next Month", Coindesk, 05/2019
  • currently transitioning from proof-of-work to proof-of-stake (PoS) blockchain
    • https://coinsutra.com/proof-of-work-vs-proof-of-stake-pow-vs-pos/
• Vitalik Buterin, recent interview "Decentralized finance is going to come first", youtube, 04/2019
  • smart contract based insurance, e.g. if a flood happens you automatically get a payout, HurricaneGuard, crop insurance (Sri Lanka)
  • social recovery approach (e.g. 5 people sharing one key)
• Ethereum Cryptocurrency: Everything A Beginner Needs To Know, coinsutra, 09/2018
  • "Ethereum is not just a blockchain; it’s a decentralized programmable blockchain-based software platform."
  • https://en.wikipedia.org/wiki/Smart_contract
• 6 Interesting Blockchain Projects

KünstlerInnenhonorare

• Linksammlung zum Thema KünstlerInnenhonorare

Other Research

Teaching & Non-Teaching

• Meet the school with no classes, no classrooms and no curriculum, Medium, 05/2019