I want to search all YouTube channels containing the keyword 'investment' either in their YouTube channel title or description, keep some channel variables and store them in a dataframe. I am using the API v3.
I created the following Python code (that loops over the different page results):
def search_channels_with_keyword(youtube, keyword): # Initialize variables for pagination (prepare the loop over YT 50 results x page) next_page_token = None channels = [] # store/append results in this list while True: # Search channels with given keyword in title/description search_response = youtube.search().list( q=keyword, part='snippet', type='channel', maxResults=50, pageToken=next_page_token ).execute() # Process the search results for search_result in search_response.get('items', []): channel_id = search_result['id']['channelId'] channel_title = search_result['snippet']['title'] channel_description = search_result['snippet']['description'] channel_thumbnailurl = item['snippet']['thumbnails']['default']['url'] channels.append({ # Append vars in list 'channels''channel_id': channel_id,'channel_title': channel_title,'channel_description': channel_description,'channel_thumbnailurl': channel_thumbnailurl }) # Check if more pages to fetch next_page_token = search_response.get('nextPageToken') if not next_page_token: break # Exit the loop if no more pages return channelsif __name__ == "__main__": keyword = 'investment' channels = search_channels_with_keyword(youtube, keyword) # Store results in pandas df df_channels = pd.DataFrame(channels) df_channelsThe above code provides some ok output (584 channels with the desired keyword 'investment'), but few manual checks let me know this is definitely not a comprehensive list. For instance, it does not provide this YT channel with +200k subscribers.
I am afraid I am missing out a lot of (important) channels. Is it an issue with the API? with my code?
Thanks all in advance,