Python
  • Blog
  • Contact
    • Overview
    Issues 0

    A curated list of the latest breakthroughs in AI (in 2023) by release date with a clear video explanation, link to a more in-depth article, and code.

    louisfb01 louisfb01 Last update: Jan 15, 2024

    2023: The best AI papers - A Review šŸš€

    A curated list of the latest breakthroughs in AI by release date with a clear video explanation, link to a more in-depth article, and code.

    With the creation of a whole new field called "Generative AI", whether you like the term or not, research hasn't slowed its frenetic pace, especially the industry, which has seen its biggest boom in implementation of AI technologies ever. Artificial intelligence and our understanding of the human brain and its link to AI are constantly evolving, showing promising applications improving our life's quality in the near future. Still, we ought to be careful with which technology we choose to apply.

    "Science cannot tell us what we ought to do, only what we can do."
    - Jean-Paul Sartre, Being and Nothingness

    Here's curated list of the latest breakthroughs in AI and Data Science by release date with a clear video explanation, link to a more in-depth article, and code (if applicable). Enjoy the read!

    The complete reference to each paper is listed at the end of this repository. Star this repository to stay up to date and stay tuned for next year! ā­ļø

    Maintainer: louisfb01, also active on YouTube and as a Podcaster if you want to see/hear more about AI!

    Twitter

    Subscribe to my newsletter - The latest updates in AI explained every week.

    Feel free to message me any interesting paper I may have missed to add to this repository.

    Tag me on Twitter @Whats_AI or LinkedIn @Louis (What's AI) Bouchard if you share the list! And come chat with us in our Learn AI Together Discord community!

    šŸ‘€ If you'd like to support my work, you can check to Sponsor this repository or support me on Patreon.

    Watch a complete 2023 rewind in 13Ā minutes

    The Full List

    Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers [1]

    Last year we saw the uprising of generative AI for both images and text, most recently with ChatGPT. Now, within the first week of 2023, researchers have already created a new system for audio data called VALL-E.

    VALL-E is able to imitate someoneā€™s voice with only a 3-second recording with higher similarity and speech naturalness than ever before. ChatGPT is able to imitate a human writer; VALL-E does the same for voice.

    InstructPix2Pix: Learning to Follow Image Editing Instructions [2]

    We know that AI can generate images; now, letā€™s edit them!

    This new model called InstructPix2Pix does precisely that; it edits an image following a text-based instruction given by the user. Just look at those amazing resultsā€¦ and that is not from OpenAI or google with an infinite budget.

    It is a recent publication from Tim Brooks and collaborators at the University of California, including prof. Alexei A. Efros, a well-known figure in the computer vision industry. As you can see, the results are just incredible.

    MusicLM: Generating Music From Text [3]

    We recently covered a model able to imitate someoneā€™s voice called VALL-E. Letā€™s jump a step further in the creative direction with this new AI called MusicLM. MusicLM allows you to generate music from a text description.

    Let's not wait any longer and dive right into the results... what you will hear will blow you away!

    Structure and Content-Guided Video Synthesis with Diffusion Models [4]

    Runway have created a system called GEN-1 that can take a video, and apply a completely different style to it in seconds. The model is a work in progress and has flaws, but still does a pretty cool style transfer from an image or text prompt into a video, something that would've been impossible a few years or even months ago. Even cooler is how it works...

    PaLM-E: An Embodied Multimodal Language Model [5]

    PaLM-E, Googleā€™s most recent publication, is what they call an embodied multimodal language model. What does this mean? It means that it is a model that can understand various types of data, such as text and images from the ViT and PaLM models we mentioned, and is able to turn these insights into actions from a robotics hand!

    Segment Anything [6]

    Segmentation - it's like the photo world's equivalent of playing detective. This superpower allows you to identify anything and everything in an image, from objects to people, with pixel-perfect precision. It's a game-changer for all kinds of applications, like autonomous vehicles that need to know what's going on around them, whether it's a car or a pedestrian.

    You also definitely know about prompting by now. But have you heard of promptable segmentation? It's the newest kid on the block, and itā€™s really cool. With this new trick up your sleeve, you can prompt your AI model to segment anything you want - and I mean anything! Thanks to Meta's incredible new SAM (Segment Anything Model), there's no limit to what you can do.

    If you're curious about how promptable segmentation and the SAM model work their magic, then you won't want to miss out on my video. In it, you'll learn all about how this amazing new technology is changing the game when it comes to image segmentation. So sit back, relax, and let me take you on a journey into the world of promptable segmentation with SAM. Trust me, you won't regret it!

    Key-Locked Rank One Editing for Text-to-Image Personalization [7]

    Imagine creating stunning Instagram images without leaving home or snapping photos! NVIDIA's new AI model, Perfusion, advances text-to-image generation with enhanced control and fidelity for concept-based visuals.

    Perfusion is a significant improvement over existing AI techniques, overcoming limitations in generating images that remain faithful to the original content. This model can accurately create these "concepts" in a variety of new scenarios.

    Perfusion builds on Stable Diffusion with additional mechanisms for locking onto and generating multiple "concepts" in new images simultaneously. This results in unbeatable quantitative and qualitative performance, opening exciting possibilities across diverse industries.

    šŸš§ While not perfect, Perfusion is a significant step forward for text-to-image models. Challenges include maintaining an object's identity and some overgeneralization, as well as requiring a bit of prompt engineering work.

    NVIDIA's Perfusion sets the stage for an exciting future of AI-generated images tailored to our desires.

    Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold [8]

    Drag Your Gan prioritizes precise object dragging over image generation or text manipulation. The AI realistically adapts the entire image, modifying the object's position, pose, shape, expressions, and other frame elements.

    šŸ¶šŸŒ„ Edit expressions of dogs, make them sit, adjust human poses, or even alter landscapes seamlessly. Drag Your Gan offers an innovative and interactive way to experiment with image editing.

    How does it work? Drag Your Gan leverages StyleGAN2, a state-of-the-art GAN architecture by NVIDIA. By operating in the feature space (latent code), the AI learns how to edit images properly through a series of steps and loss calculations.

    Even though the results are fantastic, as you will see below, it's essential to note that Drag Your Gan has some limitations, including only being able to edit generated images for now. Images are part of the distribution. Other limitations are that the selection of points is based on pixel colors and contrast, so you cannot really drag anything. If you take a part of a red car and move it staying on the red car, it might not understand that you move it at all.

    Can't wait to try it out? The authors mention that the code should be available in June. Tune in to the video (or article) to learn more about this new image manipulation style with DragYourGan!

    Check out the What's AI podcast for more AI content in the form of interviews with experts in the field! An invited AI expert and I will cover specific topics, sub-fields, and roles related to AI to teach and share knowledge from the people who worked hard to gather it.

    Neuralangelo: High-Fidelity Neural Surface Reconstruction [9]

    Neuralangelo is NVIDIA's latest breakthrough in image-to-3D AI. This new approach builds upon Instant NeRF, enhancing surface quality and providing highly realistic 3D scenes from simple images in just seconds.

    Neuralangelo aims to overcome the limitations of its predecessor, Instant NeRF, such as the lack of detailed structures and a somewhat cartoonish appearance of the AI-generated 3D models.

    The secret behind Neuralangelo's improvements lies in two key differences: using numerical gradients for computing higher-order derivatives, and adopting a coarse-to-fine optimization on the hash grids controlling levels of detail, which we dive into in the video.

    This optimization process results in a smoother input for the 3D model reconstruction, allows more information to be blended, and creates a perfect balance between consistency and fine-grain details for a realistic outcome.

    The quality of Neuralangelo's 3D models is truly astounding, but the AI does face challenges with highly reflective scenes. Nonetheless, its potential real-world applications are vast and exciting!

    TryOnDiffusion: A Tale of Two UNets [10]

    In this week's episode I decided to explore a new research called TryOnDiffusion, presented at the CVPR 2023 conference. This innovative approach represents a significant leap forward in realistic virtual try-on experiences. By training AI models to understand input images, differentiate clothing from the person, and combine information intelligently, TryOnDiffusion produces impressive results that bring us closer to the ultimate goal of a perfect virtual try-on.

    If you're intrigued by the intersection of AI and fashion, join us as we unravel the inner workings of TryOnDiffusion and its potential impact on the future of online shopping. Whether you're an AI enthusiast, a fashion lover, or simply curious about the latest technological advancements, the video offers valuable insights into the cutting-edge world of virtual clothing try-on.

    We will dive into the world of diffusion models, UNets, and attention, where all those incredibly powerful mechanisms combine forces with helping the field of fashion and online retail. Of course, this work has limitations, but (as you will see) the results are just mind-blowing and very promising.

    StyleGANEX: StyleGAN-Based Manipulation Beyond Cropped Aligned Faces [11]

    Letā€™s talk about the AI models that take your face and can transform it into a funny cartoon, edit facial attributes like changing your hair color, or simply upscale your image to make it more HD. If youā€™ve been following my articles, you know that most of these applications rely on a single model and its multiple versions called StyleGAN, which I covered numerous times already. StyleGAN is a GAN-based architecture developed by NVIDIA that can take an input and transform it into another one following a specific given style it was trained on. Itā€™s also open source, meaning that everyone can use and build on it, and why all the research papers are using it.

    The problem with StyleGAN is that it is limited to cropped and aligned faces at a fixed image resolution from the data it was trained on. Meaning that for images of the real world, you need other approaches to find the face, crop it out, and re-orient it, and it also must have the same image resolution. This is a big problem since you usually want to have high-quality images but training with them would be incredibly long.

    So what we typically do is we use the StyleGAN architecture to make the style transfer of our image, and then we use another network to upscale the image to a higher resolution. While this approach works well, itā€™s definitely not ideal. You need two models instead of one, adding more biases and potential errors, as well as needing to train both and limiting the generalizability capabilities. Fortunately for us, some amazing researchers are working on this limited input image problem and have recently published a new approach at ICCV 2023 called StyleGANEX through some very clever small changes...

    Twitter Tag me on Twitter @Whats_AI or LinkedIn @Louis (What's AI) Bouchard if you share the list!

    3D-LLM: Injecting the 3D World into Large Language Models [12]

    We've witnessed the remarkable capabilities of large language models (LLMs), but there's been a gapā€”a missing piece in their understanding of the world around us. They've excelled with text, code, and images, yet they've struggled to truly engage with our reality. That is, until now. Here's a groundbreaking leap forward in the AI landscape: 3D-LLM.

    3D-LLM is a novel model that bridges the gap between language and the 3D realm we inhabit. While it doesn't cover the entirety of our world, it's a monumental stride in comprehending the crucial dimensions and text that shape our lives. As you'll discover in the video, 3D-LLM not only perceives the world but also interacts with it. You can pose questions about the environment, seek objects or navigate through spaces, and witness its commonsense reasoningā€”reminiscent of the awe-inspiring feats we've experienced with ChatGPT.

    Even more interestingly, the authors harnessed ChatGPT's prowess to gather data through three distinct methods you'll learn about, creating a comprehensive repository of tasks and examples for each scene used to train the model...

    METAGPT: META PROGRAMMING FOR MULTI-AGENT COLLABORATIVE FRAMEWORK [13]

    This work introduces a novel framework for orchestrating large language models to work cohesively while mitigating the risks of hallucinations. This approach combines the power of AI agents with the clarity of standardized operating procedures, ensuring that the agents collaborate effectively and stay aligned with user objectives.

    Subscribe to my weekly newsletter and stay up-to-date with new publications in AI for 2023!

    Visual Instruction Tuning [14]

    Liu et al. used GPT-4 to create a general-purpose language vision model called LLaVA, the first general-purpose model that understands and follows visual and language-based instructions. Yes, they didn't use GPT-4 as the base model, but to train their model! As we will see in the video, GPT-4 was used to generate a large and high-quality dataset to train a new model that understands images. Oh and obviously it not only understands images but also text (there's the multimodality), which means it can answer a wide variety of questions about them! Learn more in the full article or in the video...

    MVDream: Multi-view Diffusion for 3D Generation [15]

    Weā€™ve seen so many new approaches to generating text, then generating images only getting better. Then, weā€™ve seen other amazing initial works for generating videos and even 3D models out of text. Just imagine the complexity of such a task when all you have is a sentence, and you need to generate something that could look like an object in the real world, with all its details. Well, hereā€™s a new one that is not merely an initial step; itā€™s a huge step forward in 3D model generation from just text: MVDream!

    Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labelling [16]

    Distil-Whisper is an audio transcription model 6 times faster than the original Whisper model, 49% smaller, and keeps 99% of the accuracy. And the best thing about it is that it is fully open-source, and you can use it right now.

    Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets [17]

    In this video, we dive into Stable Video Diffusion (SVD), exploring how this innovative technology from Stability AI is revolutionizing AI-driven video creation. Understand the core principles of diffusion models and their applications in text-to-video and multi-view synthesis, ideal for AI and digital media enthusiasts eager to grasp the future of video generation.

    If you would like to read more papers and have a broader view, here is another great repository for you covering 2022: 2022: A Year Full of Amazing AI papers- A Review and feel free to subscribe to my weekly newsletter and stay up-to-date with new publications in AI for 2023!

    Twitter Tag me on Twitter @Whats_AI or LinkedIn @Louis (What's AI) Bouchard if you share the list!

    Paper references

    [1] Wang, C., Chen, S., Wu, Y., Zhang, Z., Zhou, L., Liu, S., Chen, Z., Liu, Y., Wang, H., Li, J. and He, L., 2023. Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers, https://arxiv.org/abs/2301.02111

    [2] Brooks et al., 2022: InstructPix2Pix, https://arxiv.org/abs/2211.09800

    [3] Agostinelli et al., 2023: MusicLM, https://arxiv.org/abs/2301.11325

    [4] Esser, P., Chiu, J., Atighehchian, P., Granskog, J. and Germanidis, A., 2023. Structure and content-guided video synthesis with diffusion models, https://arxiv.org/abs/2302.03011

    [5] Driess, D., Xia, F., Sajjadi, M.S., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T. and Huang, W., 2023. Palm-e: An embodied multimodal language model, https://arxiv.org/abs/2303.03378

    [6] Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y. and DollƔr, P., 2023. Segment anything, https://arxiv.org/abs/2304.02643

    [7] Tewel, Y., Gal, R., Chechik, G. and Atzmon, Y., 2023. Key-locked rank one editing for text-to-image personalization, https://arxiv.org/abs/2305.01644

    [8] Pan, X., Tewari, A., LeimkĆ¼hler, T., Liu, L., Meka, A. and Theobalt, C., 2023. Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold, https://arxiv.org/abs/2305.10973

    [9] Li, Z., MĆ¼ller, T., Evans, A., Taylor, R.H., Unberath, M., Liu, M.Y. and Lin, C.H., 2023. Neuralangelo: High-Fidelity Neural Surface Reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 8456-8465), https://arxiv.org/abs/2306.03092

    [10] Zhu, L., Yang, D., Zhu, T., Reda, F., Chan, W., Saharia, C., Norouzi, M. and Kemelmacher-Shlizerman, I., 2023. TryOnDiffusion: A Tale of Two UNets. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4606-4615), https://arxiv.org/abs/2306.08276

    [11] Yang, S., Jiang, L., Liu, Z. and Loy, C.C., 2023. StyleGANEX: StyleGAN-Based Manipulation Beyond Cropped Aligned Faces. arXiv preprint arXiv:2303.06146.

    [12] Hong, Y., Zhen, H., Chen, P., Zheng, S., Du, Y., Chen, Z. and Gan, C., 2023. 3d-llm: Injecting the 3d world into large language models. arXiv preprint arXiv:2307.12981.

    [13] Hong, S., Zheng, X., Chen, J., Cheng, Y., Zhang, C., Wang, Z., Yau, S.K.S., Lin, Z., Zhou, L., Ran, C. and Xiao, L., 2023. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352.

    [14] Liu, H., Li, C., Wu, Q. and Lee, Y.J., 2023. Visual instruction tuning. arXiv preprint arXiv:2304.08485.

    [15] Shi, Y., Wang, P., Ye, J., Long, M., Li, K. and Yang, X., 2023. Mvdream: Multi-view diffusion for 3d generation. arXiv preprint arXiv:2308.16512.

    [16] Gandhi, S., von Platen, P. and Rush, A.M., 2023. Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labelling. arXiv preprint arXiv:2311.00430.

    [17] Blattmann et al., 2023: Stable Video Diffusion. https://static1.squarespace.com/static/6213c340453c3f502425776e/t/655ce779b9d47d342a93c890/1700587395994/stable_video_diffusion.pdf

    Tags:

    Owner

    louisfb01

    louisfb01
    GitHub Repository https://github.com/louisfb01
    Umami Analytics Client for Python

    See the usage example below for the Python API around these features. Enjoy.

    mikeckennedy
    51 Apr 18, 2024
    New generation opensource data stack

    This repository contains Docker compose script that creates opensource data analytics stack on your local machine. Currently, the stack consists of multiple components: I plan to add more components ā€¦

    zsvoboda
    52 Apr 12, 2024
    Climate Analytics using Deep Neural Networks in Python.

    ClimateNet is a Python library for deep learning-based Climate Science. It provides tools for quick detection and tracking of extreme weather events. We also expose models, data sets and metrics to jā€¦

    andregraubner
    53 Apr 17, 2024
    The purpose of this repository is to make it as easy as possible to develop and use awesome Panel extensions.

    We want to We provide You can install and use the package as simple as. Explore the sample apps Click one of the buttons Thanks I would love to support and receive your contributions. Thanks.

    awesome-panel
    54 Apr 16, 2024
    This project helps me to understand the core concepts of Apache Airflow. I have created custom operators to perform tasks such as staging the data, filling the data warehouse, and running checks on the data quality as the final step. Automate the ETL pipeline and creation of data warehouse using Apache Airflow. Skills include: Using Airflow to automate ETL pipelines using Airflow, Python, Amazon Redshift. Writing custom operators to perform tasks such as staging data, filling the data warehouse, and validation through data quality checks. Transforming data from various sources into a star schema optimized for the analytics teamā€™s use cases. Technologies used: Apache Airflow, S3, Amazon Redshift, Python.

    For this project, I have build four different operators that will stage the data, transform the data, and run checks on data quality. You will also need to have software installed to run and execute ā€¦

    ddgope
    57 Apr 25, 2024
    Utility functions for "Data Mining for Business Analytics: Concepts, Techniques, and Applications in Python"

    Collection of utility functions used in the book Should this not work, for example when you are behind a firewall, download the package from pypi and install from file, e.g. Adjust the link as requirā€¦

    gedeck
    58 Apr 25, 2024
    :earth_americas: Analyze and generate reports of web logs (NGINX)

    Lala is a Python library for access log analysis. It provides a set of methods to retrieve, parse and analyze access logs (only from NGINX for now), and makes it easy to plot geo-localization or timeā€¦

    Edinburgh-Genome-Foundry
    60 Jun 20, 2023
    Exploratory data analysis projects on Different datasets to enhance Data Analysis and visualization
    shsarv
    61 Apr 07, 2024
    A simple package to abstract away the process of creating usable DataFrames for data analytics. This package is heavily inspired by the amazing Python library, Pandas.

    A simple package to abstract away the process of creating usable DataFrames for data analytics. This package is heavily inspired by the amazing Python library, Pandas. Utilize the CreateDataFrame funā€¦

    kfultz07
    64 Apr 26, 2024
    A python project starter template for data-analytics and data-science.

    A project template to quick start data analytics and machine learning task. You can connect with me here:

    janishar
    70 Apr 01, 2024

    Subscribe to our newsletter