This post is republished from it's original on my personal site, here: dedovic.com/writings/reverse-zoology.
Update March, 2022: Ellie and I will be participating in the Bright Moments Berlin event in April! Minting will be done in-person and via surrogate. If you’re able to, come out and support us! More info at: brightmoments.io/cryptoberlin
A few months ago I was introduced to Ellie Pritts, an outstanding artist out of L.A. who is popular in the NFT and digital art community. She was seeking a technical partner to work on a project, and I needed someone who can navigate art as a business. Our goals and personalities lined up pretty well, so, we decided to partner on what has now become the Reverse Zoology project.
It’s been a few months since we first chatted, and a lot of progress has been made. My goal here is to talk through some technical details, how the art works, and the numerous pitfalls I encountered along the way. I imagine there is more to discuss than can be done in a single post, but I have to begin somewhere.
An Overview
Reverse Zoology is an evolution of frogeforms with each piece created by the same process, parameterized.
For a single artwork, it looks something like this:
Select 3 source images from a collection provided by Ellie
Visually morph between the images, slowly changing
Warp the outcome from step 2, making it wavy or distorted
Composite the warped video from step 3 onto a background video
Ellie was responsible for generating the source images (Step 1) using a complex, psychedelic analog process, GANs, and much more! But that’s beyond the scope of my work and you can read about it in this Twitter thread.
Morphing
To visually morph between the images, I attempt to replicate an effect known as a Morph Cut or Smooth Cut.
The Morph Cut is a video transition effect that comes as a stock effect in many video editing softwares, such as Adobe Premier Pro and DaVinci Resolve. It is typically used to remove small sections of interviews, for example between unwanted pauses or stutter. In Adobe Premier Pro, according to their documentation, this is achieved using a combination of “face tracking and optical flow interpolation to create a seamless transition” between sections of video. DaVinci Resolve takes a similar approach, using optical flow processing, to perform the morphing effect. This transition becomes interesting when stretching it over the recommended range, of a few frames, to something much longer, like 10 or more seconds.
But how do these effects actually work? To answer that, we first need to look at a video morphing effect. My favorite use of morphing, and one of the earliest, come the amazing sequence at the end of Michael Jackson’s Black or White music video. See it happen around the 5:27 mark.
To achieve the face morphing effect, a transformation between two faces is performed where important features (eyes, nose, mouth, etc.) are moved and stretched from one person to the other. Next, the colors of individual pixels must be mixed to seamlessly transition over the duration of the effect.
A Custom Morphing Effect
This is where optical flow processing comes into play. Optical flow is a description of the motion of high-level features between two frames of video. For example, imagine a video of a basketball. The optical flow between two consecutive frames should tell us the ball is moving down while the background is static. Maybe a bird is somewhere in the shot, and that has its own motion. These are all high-level features, not individual pixels or edges.
Using an optical flow algorithm should allow me to predict the motion of things between two frames of video, or, if I feed it two of Ellie’s GAN animals, the optical flow between the two may yield some interesting results. The hope is mapping visually similar features and morphing it in an interesting way.
Luckily for me, OpenCV has a handful of (mostly) out-of-the-box optical flow processing algorithms. The output of these algorithms is a vector field, telling me where the algorithms think regions of the first image move to line up on the second image.
The morphing effect now looks something like this:
Get two source images, A and B
Calculate the optical flow fields
Flow AB = Image A → Image B
Flow BA = Image B → Image A
For every pixel in Image A
Move it based on Flow AB
Change the color, mixing between Image A and Image B
Repeat for every pixel in Image B, using Flow BA
So… what does that look like?
Awesome! It’s not the same as Premier Pro but its unique and I can parameterize the effect to fit the content.
Warping
Now that morphing is out of the way, the next step is to warp the video, starting by defining warping. Warping is a distortion effect on a video. Something like the effect of ripples of water or of looking through a thick piece of glass. It means the light coming into my eyes isn’t traveling a straight path, its bent, bounced, blocked along the way. So, why does that happen, in the case of thick, abnormal glass?
Light refracts. If a ray of light travels through a medium of different density or composition as air, it will not travel a straight path. That’s why a straw or pen looks weird in a glass of water, like the image below.
Water causes refraction in what we see because of the physics of light. It follows well-known rules, and, more importantly, it follows well documented rules. Rules that computer graphics folk have already figured out and implemented - meaning I can simulate light refraction without having to reinvent too much.
So let’s describe a system for warping an image. We have:
a coordinate system. X is left-right, Y is up-down, Z is in-out
a camera, somewhere on the Z axis, looking in
an image sitting on the XY plane, looking out to the camera
every pixel of my screen is mapped 1:1 to a pixel on the image
That’s easy, I am displaying an image without modification. But now, let’s add a glass material. This will be a simple differentiable height map, meaning I can get the height of the glass and its normal, i.e., the 3D angle of the surface, by specifying an X and Y coordinate.
Now, updating the warping system from below, we must:
calculate how far a ray of light travels before it hits the glass
find the angle between the ray of light and the surface of the glass
use Snell’s law to determine the angle of refraction based on the properties of the glass and the air
measure how long the ray of light will travel, in its new direction, before hitting the image
determine the pixel which is hit by the refracted ray of light
Now we have a physically based warping effect!
Putting it all Together
I will cover this section in more detail in a follow up post. Feel free to contact me if there is something you want me to discuss or detail!
Tools, Languages, and Frameworks
I chose to develop the main processing application in Rust. I wanted a performant language with fine-grained control over parallel programming, which excluded Python, Javascript, and Ruby. Not only that, but I also needed access to many low-level libraries and APIs, opting not to use a JVM language. Rust is fun and I find C++ to be verbose and dangerous.
For managing dependencies, the Nix package manager seemed like a great fit. I’m writing code on my Mac and need to have my partner’s laptop working without issue. Installing OpenCV, GStreamer, Rust, etc. on a phone call was a non-option. But a simple git clone && nix-shell could work. I did have some issues making the main codebase function on both Mac and Linux, and I don’t know if I’ll ever figure that out. Nix is simply not friendly when it comes to graphics applications.
For optical flow processing, the heavy lifting is done by OpenCV. For high-performance morphing, warping, and compositing, I wrote OpenCL kernels. I started the project writing OpenGL shaders but realized this isn’t ideal. I don’t need any OpenGL context since I will be writing compute shaders, I’m essentially bastardizing the framework for my use case. OpenCL is smoother sailing.
For the UI, I decided on Nannou and Conrod. This was a mistake, but I’m living with it. Nannou is a good project, but not well suited for building UIs that are decoupled from some underlying computation. What makes it even more difficult the Conrod project - it is not maintained and a headache to use. If I could rewrite everything from scratch, I would stick to a client-server model, possibly communicating over TCP or HTTP.
Closing Remarks
Overall, this has been one of the biggest projects I’ve embarked on. I made numerous mistakes and learned many hard lessons. It is the single biggest Rust project I’ve worked on, bumping against many rough edges of the language. I have to say, I’m a Rust evangelist now. And a Nix evangelist too.
The next few topics I think I’ll cover are:
Glass warping, in detail, with some actual code
OpenCL kernels and performance considerations
My GStreamer based video frame grabber
Thoughts on using Rust, OpenCV, OpenCL, and Nix together to create a (mostly) cross-os development environment
Stevan Dedovic is a software engineer and visual artist based out of Chicago. His focus is on computation expressionism, high performance software, and sharing what he learns along the way. You can follow his work through Instagram, @mr_praline, or on his personal site, dedovic.com.
If you’re looking for fast publication and low fees, here’s a summary of your options based on the journals and indexing services you’re considering:
IJSET (International Journal of Scientific Engineering and Technology):
IJSRET (International Journal of Scientific Research and Engineering Trends):
IJRTSSH (International Journal for Research Trends in Social Science & Humanities):
ABCD Index:
Role: The ABCD Index is an indexing platform, not a journal. It can provide visibility for your published paper if your chosen journal is indexed here.
If you're looking for journals with low publication fees, here are some recommendations across disciplines, including the ones you're interested in:
Are you struggling with complex IT concepts or tight assignment deadlines? Our Information Technology Assignment Help service is designed to provide expert guidance and top-notch solutions for students pursuing IT courses. Whether it's programming, database management, networking, cybersecurity, or cloud computing, our team of experienced professionals ensures well-researched, accurate, and plagiarism-free assignments.
Why Choose Us?
Expert Assistance: Skilled IT professionals with academic expertise.
Timely Delivery: Submit your assignments before the deadline.
Affordable Pricing: High-quality services within your budget.
24/7 Support: Round-the-clock assistance for all your queries.
Visit Here
Get the grades you deserve with our comprehensive and reliable IT assignment help. Visit here to know more and transform your academic journey today!
Professional Nursing Writing Tailored to Your Unique Needs
In the healthcare field, nursing is one of the most demanding professions, requiring both practical and theoretical knowledge to provide optimal patient care. Nursing students face the challenge of balancing clinical practice with academic requirements, which can often lead to stress, burnout, and difficulty in completing assignments. The ability to write well-researched, clear, and insightful nursing papers is an essential skill for success in nursing programs. However, given the heavy workload and rigorous study schedules, many students turn to professional nursing writing services for assistance. These services offer customized, high-quality papers that are tailored to meet each student's specific needs, ensuring that they meet academic standards while allowing students to focus on…
If you're looking for journals that offer fast-track publishing for research papers, here are some tips and resources:
1. Fast-Track Journals
Many journals provide expedited review and publication processes for authors who need their work published quickly. These options often involve additional fees or strict manuscript requirements. Below are some suggestions:
IJSRET (Engineering & Technology)
Ijrtssh ( Open Access)
Fast-Track Journals (ijrtssh)
ABCD Index (Multidisciplinary)
Ijset Journals (n fast publication timelines)
Ijnrefm (Various Disciplines, Open Access)
Actionable Steps
Identify Your Target Journal: Select a reputable journal in your field with a fast publication process.
Contact Editors: Email the editor to inquire about expedited options if they're not explicitly listed.
Ensure Readiness: Ensure your manuscript is error-free and formatted according to the journal's…
https://forumweb.hosting/members/cafosi5225.31666/#about
https://www.miguelassis.com/group/mysite-231-group/discussion/eff296b2-d84a-42c2-b5c2-d218edff7280
https://trabajo.merca20.com/author/letog51430/
http://www.biznas.com/Biz-postsm339389_-Emergency-Nursing--Challenges-and-Rewards-in-Critical-Care.aspx#post339389
https://freshsites.download/socialwow/upload/files/2024/07/fTIQ1v5rYoBWJ2cQDVlK_30_7132e3503228007a640fb83f710b3558_file.pdf
https://mellrakforum.hu/forums/users/katovoh367/
https://vadea.viaafrika.com/forums/users/nogeh20722/
https://lyfepal.com/forums/thread/3099/Pediatric-Nursing-Specialized-Care-for-Young-Patients
https://bresdel.com/blogs/626348/Mental-Health-Nursing-Addressing-the-Growing-Need
https://charismaticplanet.com/author/dayojo7766/
https://www.hanaromartonline.com/forum/main/comment/492439fe-7ae5-4908-82ab-d10590d2d327?postId=63f363a564ec0c0012294d19
https://news.bangboxonline.com/the-impact-of-nursing-writing-services-on-career-advancement
https://firstamendment.tv/read-blog/52733
https://www.bhimchat.com/1722352258140517_51438
https://www.seereadshare.com/author/dayojo7766/
https://www.design-buzz.com/author/dayojo7766/
https://www.telix.pl/profile/dayojo7766/
https://coolmomsrule.com/author/dayojo7766/
https://hikyou.jp/forums/users/habed89499/
https://www.pmimauritius.com/forum/main/comment/fe65553b-a596-4d98-9e00-c725cc632b73?postId=6426888fb5e645001266ca97
https://www.tatasechallenge.org/community/forums/users/dayojo7766/
https://animationpaper.com/forums/users/nogeh20722/
https://www.odeh.ps/wall/blogs/794/The-Future-of-Nursing-Writing-Services-Trends-and-Innovations
https://directory.womengrow.com/author/dayojo7766/
https://targetshooting.ca/forums/users/dayojo7766/
https://www.clearnotebooks.com/ja/authors/6879739
https://projectnursery.com/author/dayojo7766/
https://muabanthuenha.com/author/dayojo7766/
https://git.bolin.su.se/bolin/keuper-wild-2020/-/issues/1409
https://truckymods.io/user/256140
https://drivehud.com/forums/users/dayojo7766/
https://legalforms.ng/ask-a-lawyer/profile/dayojo7766/
http://mehfeel.net/mehfeel/blogs/post/29225
https://bavave.com/wp-content/uploads/2024/07/Untitled-document-10.pdf
https://www.hituponviews.com/wp-content/uploads/2024/07/Untitled-document-11-1.pdf
https://www.wrkz.work/thread-3-post-4072.html#pid4072
https://git.ctdo.de/interfisch/hydroponic-controller/issues/7#issue-1724
https://photozou.jp/user/top/3369193
https://forum.fractalfuture.net/t/wann-werden-ki-roboter-schlau-er-als-der-mensch/2085/897?u=dayojo7766
https://bustedwallet.com/author/dayojo7766/
http://demo.advised360.com/read-blog/164468
https://kambadyami.net/upload/files/2024/07/4vuolE88UPMm67x1qCxR_31_290e8832291135fecf23580239e585aa_file.pdf
https://www.pnc-contact.com/forums/users/dayojo7766
https://phoenixsunsclub.com/read-blog/5864
https://www.surajmani.in/forum/main/comment/cc2ae09a-29a5-433e-88c9-3a8c238eb44f?postId=630821b42046ce00128dccc6
https://www.blackswancountryclub.com/forum/main/comment/105d92bb-ad6b-4124-8165-8ad65d9a9e50?postId=66aa65e2a23d670011b9c483
https://gitlab.com/-/snippets/3733747
https://git.project-hobbit.eu/-/snippets/26034
https://www.frankentoon.com/profile/katovoh367/profile
http://logebiv306.stranky1.cz/nova-stranka-328123/
https://www.mmicc.org/forum/main/comment/b0105693-5f87-4aca-a3bf-f22a8f3375dd?postId=6352717209d83d00128a393f
https://www.lidinterior.com/interioworld/main/comment/8fe18f52-2c3e-46eb-94e9-119cdcd77b2d?postId=6155856419c1e90016758949
https://www.cloudtenpictures.com/forum/main/comment/8ac0b444-baad-47f2-9d76-437b796114a2?postId=6446c6a0a7cf760012bdee20
https://www.broadwaychurchkc.org/forum/main/comment/84790e01-3c84-4cd5-8fca-69268461735a?postId=63ca79543421a1001218670e
https://www.exoltech.ps/blogs/post/61016
https://www.academicexperts.org/discussions/20134/?c=186446
https://tutorsonspot.com/questions/from-theory-to-practice-applying-academic-knowledge-with-the-help-of-nursing-writing-services-8xsz/
https://www.vgnetwork.com/members/vavohik964/profile/
https://linguacop.eu/blog/the-economic-impact-of-nursing-writing-services-on-healthcare-institutions/
https://forumforex.id/forum/forumforexid/rule-informasi/59314-effective-communication-strategies-in-nursing-writing-services
https://www.ictdemy.com/assembly-forum/assignment-help-company-in-australia-case-study-help-62c4195d2ea8d?page=1#goto10495
https://heroes.app/blogs/432295/The-Benefits-of-Editing-and-Proofreading-Services-for-Nursing-Papers
https://empowering-nursing-profe-pqddsaq.gamma.site/
https://www.debwan.com/blogs/563649/How-Nursing-Writing-Services-Can-Help-with-Complex-Case-Studies
https://www.mindspec.org/author/dayojo7766/
http://socialnetwork.linkz.us/blogs/78327/From-Theory-to-Practice-How-Nursing-Paper-Writing-Services-Aid
https://www.inventoridigiochi.it/membri/civalor491/
https://airsoftc3.com/article/8118/The-Future-of-Nursing-Writing-Services-Trends-and-Predictions
https://cafosi5225.diary.ru/
https://www.fundable.com/wayoli2661-wayoli2661
https://myjoye.com/forums/topic/11893/professional-development-through-nursing-writing-services/view/post_id/309085
https://www.saprec.org/forum/main/comment/d7c8c90a-cb22-44ea-b5f5-683caeaf18c5?postId=5fa91c1af6df54002d9c86e2
https://dawlish.com/user/details/22153
https://forum.gettinglost.ca/user/katovoh367
https://www.forum.motokobiety.pl/wiad602333.html#602333
https://www.gemresearchuk.com/forum/main/comment/f3ae2491-d7e5-423a-bf0e-6b7d9de07c96?postId=643305580c258a00135f4d14
https://thecaptivestory.com/community/profile/civalor491/
https://war-lords.net/forum/thread-937-post-138481.html#pid138481
http://share.pkbigdata.com/orvillegoldsch/replica-bags1986/issues/1
https://ivalor491itivelycom.livepositively.com/overcoming-writer-s-block-in-nursing-writing/new=1
https://writefreely.ethibox.fr/civalor491/span-style-text-decoration-underline-em-strong-andnbsp-using
https://www.whizolosophy.com/category/activity-fitness-sport/article-column/writing-reflections-in-nursing-a-guide-for-students
https://notepadonline.co/share/jy51f91722685508
https://forum.goldenear.com/forum/user/14708-civalor491
https://dictanote.co/n/1046579/
https://www.themoviedb.org/u/civalor491
https://www.realestatedepot.com/forums/discussion/general/join-the-best-aws-cloud-course-in-pune-for-career-growth
https://www.robertehall.com/profile/logebiv306/profile
https://aboutnursepractitionerjobs.com/author/habed89499/
https://www.keiteq.org/forum/main/comment/e075f12c-50fb-4211-86af-2bd28d770055?postId=5f7d862f4cbeaa00b9ba04b2
https://preview.shorthand.com/cAUmQhCY21vxFCsc/responsive/desktop
https://v8.next.forgejo.org/civalor491/treert/issues/1#issue-495
https://gratitudehabitat.com/forums/topic/have-you-used-authors-or-services-to-write-literature-reviews/page/8/#post-1329428
https://amazingradio.com/profile/civalor491
https://www.weddingbee.com/members/civalor491/
https://www.victoria-restaurant.net/wall/blogs/1255/Ethical-Writing-in-Nursing-Guidelines-and-Responsibilities
https://www.aegeanonline.edu.gr/wall/blogs/874/Writing-Tools-and-Resources-for-Nursing-Professionals
https://www.cowcow.com/user/civalor491-3203300/profile
https://www.cdt.cl/user/civalor491civalor491/
https://www.lumberjocks.com/members/civalor491.394675/#about
https://staging.partners.org/myapnea.org/members/civalor491/posts