No change. Just stimulated by other people's comments. I see a lot of forecasters cite additional improvement/learning by AlphaGo. It's interesting what the learning rates of AlphaGo is. As I understand it, on chess computers, the improvements mainly came from throwing more processing power at the program.
For Deepminds, like AlphaGo, what do you think the constraints are on its learning rates? Is it just more time for it to play itself? Get a bigger collection of past games? Is it time for the engineers to analyze the data and tweak the neural net architecture? Is it still throwing more CPUs at the problem?
The constraints are that they are still very complicated cuckoo clocks. The moment it is not about sheer power, humans have the edge for the same reason humans recognize faces (and tiny irregularities in images) much better than computers.
It appears that neural network part won't benefit that much from scaling up. The tree search is directly scalable. The software is distributed, so theoretically they can make it much larger.
I just asked a friend who is a computer scientist. He told me that massive parallelization of neural networks is an active area of research and, in general, it does not work particularly well. Something about everything being connected to everything and the need for individual cores to exchange the results.
@GJDrew How much did you think about this AlphaGo prediction? Would you be willing to wager money on Lee Sedol in a bet? If so, at what odds?
I am trying to figure out how much I can learn from your low 25% prediction given your superforecaster status. You are clearly great at forecasting, but maybe in this particular case you didn't think as carefully about it as some of the other people who are confidently estimating that AlphaGo is most likely to win (?), so I'm not sure how much I should believe your low estimate and lower mine from what it otherwise would be. Thanks.
Looks like you were right @WJK. I think I under-estimated how good the Go computers were getting even before this Deep Mind advance. I was using base rates from 10-15 years ago.
@GJDrew I raised my prediction steadily over time up to 70% on March 1st. But I think the whole time I was deliberating on this I was still biased by my initial view that "It will be quite difficult to improve by the large amount necessary to reach Lee Sedol's skill level." Really I don't think I had any good reasons to believe this other than very weak ones.
No change. Just stimulated by other people's comments. I see a lot of forecasters cite additional improvement/learning by AlphaGo. It's interesting what the learning rates of AlphaGo is. As I understand it, on chess computers, the improvements mainly came from throwing more processing power at the program.
For Deepminds, like AlphaGo, what do you think the constraints are on its learning rates? Is it just more time for it to play itself? Get a bigger collection of past games? Is it time for the engineers to analyze the data and tweak the neural net architecture? Is it still throwing more CPUs at the problem?
The constraints are that they are still very complicated cuckoo clocks. The moment it is not about sheer power, humans have the edge for the same reason humans recognize faces (and tiny irregularities in images) much better than computers.
Yea but they do improve over time. So what's the driver of the improvement I wonder?
It appears that neural network part won't benefit that much from scaling up. The tree search is directly scalable. The software is distributed, so theoretically they can make it much larger.
How did you get to the first part, the the NN won't benefit from scaling up? Is that from a link?
I just asked a friend who is a computer scientist. He told me that massive parallelization of neural networks is an active area of research and, in general, it does not work particularly well. Something about everything being connected to everything and the need for individual cores to exchange the results.
@GJDrew How much did you think about this AlphaGo prediction? Would you be willing to wager money on Lee Sedol in a bet? If so, at what odds?
I am trying to figure out how much I can learn from your low 25% prediction given your superforecaster status. You are clearly great at forecasting, but maybe in this particular case you didn't think as carefully about it as some of the other people who are confidently estimating that AlphaGo is most likely to win (?), so I'm not sure how much I should believe your low estimate and lower mine from what it otherwise would be. Thanks.
Looks like you were right @WJK. I think I under-estimated how good the Go computers were getting even before this Deep Mind advance. I was using base rates from 10-15 years ago.
@GJDrew I raised my prediction steadily over time up to 70% on March 1st. But I think the whole time I was deliberating on this I was still biased by my initial view that "It will be quite difficult to improve by the large amount necessary to reach Lee Sedol's skill level." Really I don't think I had any good reasons to believe this other than very weak ones.