The best visual explainer for Reinforcement Learning. 🤣 This one got a big set of RL concepts covered: exploration, replay buffer, imitation learning, sparse reward, distributed training, parallel rollouts, multi-agent, and maybe, just maybe ... the mythical Q* 🤣

#Artificial Intelligence #Whattup Doggy 🐾 #For Teachers By Teachers 

DANK MEMES

Artificial Intelligence

Whattup Doggy 🐾

For Teachers By Teachers

The best visual explainer for Reinforcement Learning. 🤣 This one got a big set of RL concepts covered: exploration, replay buffer, imitation learning, sparse reward, distributed training, parallel rollouts, multi-agent, and maybe, just maybe ... the mythical Q* 🤣  #Artificial Intelligence #Whattup Doggy 🐾 #For Teachers By Teachers

Sam Mendoza

A lot of lessons to be learned here. 1. You have to keep trying if it’s something you actually desire. 2. Just because you fall once or twice doesn’t mean you weren’t on the right track. 3. When you see someone reaping the rewards you desire, don’t just try to jump to the end where they are. Follow the path they did, it’s more efficient that way (and you also learn more along the way).

Nate

wouldn’t their sense of smell form a natural policy gradient?

the dogs can clearly see (and smell) how close they are to the food, that's the guiding reward. But they have to escape the local maximum of trying to jump onto the table, and take the circuitous route instead.

I'm guessing that "reward signal" and "policy gradient" are forms of intermediate rewards. Since this video is a one-epoch example, the dogs would need some indication that climbing the ramp was a good move. It seems to me that they very quickly climbed the ramp so had past knowledge (of course) of using a ramp to get higher - where the rewards awaited them.

You don't need immediate rewards. An example is Alpha zero, where Google trained a model to behave as an evaluation function for a Monte Carlo Tree search based, chess engine. The Policy gradient method can also work if the reward is sparse (it should not be too sparse though).The only reward signal was the game's outcome ( positive reward for a win, negative for a loss and zero for a draw). And because they used Monte Carlo Tree search, the entire game would have to be played out in random during the initial training iterations to generate a reward signal! And this should go on for multiple epochs.The training algorithm will rely on the outcome of a randomly played-out chess game (during the initial stages of the training process, in which the weight are random) and use the policy gradient method to update the evaluation function. They didn't have a reward signal leading all the way to the game's final outcome. If they did, they will need a chess engine to generate them! But doing so defeats the purpose of creating a self-learned chess engine.

Jerome Cody

Cyril

wouldn’t their sense of smell form a natural policy gradient?  the dogs can clearly see (and smell) how close they are to the food, that's the guiding reward. But they have to escape the local maximum of trying to jump onto the table, and take the circuitous route instead.

Todd Smith

And it's worth watching the little dog in light pink who was the first to get the right idea. She was so polite and patient with the other dogs but she got a little discouraged. I'm so glad that she got to share in the rewards!

Sam Mendoza

A group of likeminded people in Artificial Intelligence are talking about this.