Kun Le (@kunlqt) header
Kun Le (@kunlqt) profile picture

Founder and CTO @PDGTechJsc, @RetailFit Mobile Ads&Marketing.Interested in any Technologies

21.05048,105.808519
Joined March 2009
Photo by Kun Le (@kunlqt)Photo by Kun Le (@kunlqt)Photo by Kun Le (@kunlqt)Photo by Kun Le (@kunlqt)Photo by Kun Le (@kunlqt)Photo by Kun Le (@kunlqt)Photo by Kun Le (@kunlqt)Photo by Kun Le (@kunlqt)Photo by Kun Le (@kunlqt)Photo by Kun Le (@kunlqt)
Kun Le retweeted
We have reached an agreement in principle for Sam Altman to return to OpenAI as CEO with a new initial board of Bret Taylor (Chair), Larry Summers, and Adam D'Angelo. We are collaborating to figure out the details. Thank you so much for your patience through this.
5,332
11,937
11,189
61,786
41,803,300
PLVS is a real-time system that leverages sparse SLAM, volumetric mapping, and 3D unsupervised incremental segmentation. PLVS stands for Points, Lines, Volumetric mapping, and Segmentation. github.com/luigifreda/plvs
11
1
76
5,716
Very happy to finally share code & models for PIPs++! Long-term tracking, for hundreds of points, in seconds. code: github.com/aharley/pips2 paper: arxiv.org/abs/2307.15055 project: pointodyssey.com
17
125
9
677
75,591
It's so simple to remove background using Python, you can reduce code lines to 5 📔 Best Python Books: bit.ly/3REqgBm #python #programming #developer #morioh #programmer #softwaredeveloper #webdev #webdeveloper #webdevelopment #pythonprogramming #pythonquiz #ai #ml
72
535
20
2,588
334,974
Kun Le retweeted
VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning paper page: huggingface.co/papers/2309.1… Although recent text-to-video (T2V) generation methods have seen significant advancements, most of these works focus on producing short video clips of a single event with a single background (i.e., single-scene videos). Meanwhile, recent large language models (LLMs) have demonstrated their capability in generating layouts and programs to control downstream visual modules such as image generation models. This raises an important question: can we leverage the knowledge embedded in these LLMs for temporally consistent long video generation? In this paper, we propose VideoDirectorGPT, a novel framework for consistent multi-scene video generation that uses the knowledge of LLMs for video content planning and grounded video generation. Specifically, given a single text prompt, we first ask our video planner LLM (GPT-4) to expand it into a 'video plan', which involves generating the scene descriptions, the entities with their respective layouts, the background for each scene, and consistency groupings of the entities and backgrounds. Next, guided by this output from the video planner, our video generator, Layout2Vid, has explicit control over spatial layouts and can maintain temporal consistency of entities/backgrounds across scenes, while only trained with image-level annotations. Our experiments demonstrate that VideoDirectorGPT framework substantially improves layout and movement control in both single- and multi-scene video generation and can generate multi-scene videos with visual consistency across scenes, while achieving competitive performance with SOTAs in open-domain single-scene T2V generation. We also demonstrate that our framework can dynamically control the strength for layout guidance and can also generate videos with user-provided images. We hope our framework can inspire future work on better integrating the planning ability of LLMs into consistent long video generation.
6
108
13
402
123,444
Great paper on Video Inpainting. ProPainter allows you to remove objects and watermarks, complete masked areas in videos, and extend any video view. The open-source model improves propagation and transformer for Video Inpainting. It's composed of three components: 1. Recurrent flow completion 2. Dual-domain propagation 3. Mask-guided sparse Transformer
9
138
11
680
87,275
Love the awesome-X collection of repos on github. They are one of the best places to find the most relevant papers on a given topic and track the evolution of research happening on it over the years. e.g. github.com/awesome-NeRF/awes…
2
12
74
9,882
Kun Le retweeted
Generative Image Dynamics paper page: huggingface.co/papers/2309.0… present an approach to modeling an image-space prior on scene dynamics. Our prior is learned from a collection of motion trajectories extracted from real video sequences containing natural, oscillating motion such as trees, flowers, candles, and clothes blowing in the wind. Given a single image, our trained model uses a frequency-coordinated diffusion sampling process to predict a per-pixel long-term motion representation in the Fourier domain, which we call a neural stochastic motion texture. This representation can be converted into dense motion trajectories that span an entire video. Along with an image-based rendering module, these trajectories can be used for a number of downstream applications, such as turning still images into seamlessly looping dynamic videos, or allowing users to realistically interact with objects in real pictures.
11
300
24
1,426
174,700
1
30
118
20,324
Treating Attention Deficit Disorder in #LLMs — @abacusAI show you how, with open-sourced context length extensions: blog.abacus.ai/blog/2023/07/… —————— #AI #MachineLearning #DeepLearning #GenerativeAI #DataScience #DataScientists #BigData #NLProc #ChatGPT xitter.gq/bindureddy/status/1686…
🌟Announcing Long Context OSS LLM - Giraffe 🌟 We are thrilled to announce 2 new open-source LLMs! Today's SOTA open-source LLMs have one big shortcoming! These LLMs have a very small context length of only 2K This translates to them not being very useful when it comes to creating a Custom LLM based on your knowledge base. You can’t send the LLM much data in a single call, which has a very negative effect on model performance. Giraffe is a Llama 1 fine tune that extends context lengths to 4K and 16K. We are open-sourcing these models, evaluation datasets, and performance experiments. These models work well for real-world applied AI systems Relevant links: Git repo: github.com/abacusai/Long-Con… Huggingface: 16k context - huggingface.co/abacusai/Long… 4k context - huggingface.co/abacusai/Long… Blog post: blog.abacus.ai/blog/2023/07/…
5
16
6,340
Kun Le retweeted
These machine learning cheatsheets contain some of the best and well-organized ML content I've come across. Sometimes, it's just good to understand the concept at a high level and it's context before going deep. This resource helps with that. stanford.edu/~shervine/teach…
31
547
8
2,569
Recommended read on @Medium: “Under the Hood of Modern Machine and Deep Learning” ift.tt/3fXQLNN
3